Research 路 NewsShorts-Bench 路 Stage 12: Translation
Does translating a news clip change its meaning? Six translators tested
Every translator keeps 90.6-94.1% of misleading-or-fair verdicts, against a 95.4% judge noise floor. Open Gemma 4 26B and DeepSeek V4.1 Flash are closest to the floor; Cloud Translation NMT flips the most fair clips to misleading (12 of 202).
- Data
- 202 translations
- Compute
- Hosted APIs
- Noise floor
- 95.4%
- Metric
- Verdict kept %
Clips whose verdict survives translation (%) 路 higher is better 路 dot = measured
A news short in English can reach far more people with Hindi or Spanish subtitles. But translation is another edit. Drop a "reportedly" or flip a negation and a fair clip starts to mislead; smooth over a contrast and a misleading one looks fine. Translation is usually scored by how close it is to a reference (chrF, COMET). We asked a different question: does the verdict survive? Is a fair clip still fair after translation, and a misleading one still misleading?
Read the full study on Every Local AI