All research

Can AI catch a news clip that misleads out of context? 10 judges tested

Result

Gemini 4 Argon leads (macro-F1 0.847). Open GLM-5.2 is within about 3 points with the fewest false alarms on fair trims. The MiniCheck fact-checker finds 2% of misleading cuts: grounding checkers can't see a context cut.

Data
162 clips
Compute
Hosted APIs
Truth
By construction
Metric
Macro-F1

A clip is an excerpt, so every word in it was really said. That is exactly why it can mislead: end a clip one sentence before "...but the court overturned that ruling" and the viewer leaves believing the opposite of the story. Fact-checking models can't catch this. They ask "is this supported by the source?", and the answer is always yes. We built 162 traps from real news to measure which judges can catch it. The standard support checker finds 2% of them.

Read the full study on Every Local AI