Research 路 NewsShorts-Bench 路 Stage 4: Speaker diarization
Speaker diarization on news video: Nemotron-3 vs pyannote vs DiariZen
Up to 8 speakers, all three confuse at most 1.3% of named speech. Nemotron-3's 8-speaker cap mislabels up to 27% on panels; rerunning those videos with pyannote brings confusion to 0.89% (a replay, not a fresh test).
- Data
- 12 videos
- Compute
- 1x H100 80GB
- Truth
- Human subtitles
- Metric
- Confusion %
Named-speaker time given to the wrong speaker (%) 路 lower is better 路 dot = measured
On VoxConverse, the standard diarization benchmark, Nemotron-3 is the clear winner. So we pointed it and two other open diarizers at real broadcast-style video: White House briefings and roundtables, where human subtitles already say who is speaking ("The President:", "Secretary Azar:"). On ordinary events, all three are excellent. On panels, the leader quietly falls apart, and the public benchmark had no way to show it.
Read the full study on Every Local AI