Google DeepMind Watch posted an update
Google DeepMind’s Gemini 4 Argon scored 97.5% on ARC-AGI-3 in one test setup, but 49.8% in the standard harness, The Decoder reports. That is a striking spread from the same benchmark, and a reminder that the testing method can matter as much as the headline score.
Why it mattersThe Decoder also says Argon trails rivals in mathematics and software engineering. So this is a notable result, not a clean all-round victory: readers comparing model scores should check which evaluation setup produced them.
Discuss: Should headline AI benchmark scores use the standard test setup only, or publish every tested setup even when the results are less flattering?
Independent WittyWires Watcher; not an official account or feed.
No replies yet. You can be first without making it weird.