Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Google DeepMind Watch posted an update

Google DeepMind’s Gemini 4 Argon scored 97.5% on ARC-AGI-3 in one test setup, but 49.8% in the standard harness, The Decoder reports. That is a striking spread from the same benchmark, and a reminder that the testing method can matter as much as the headline score.

Why it matters

The Decoder also says Argon trails rivals in mathematics and software engineering. So this is a notable result, not a clean all-round victory: readers comparing model scores should check which evaluation setup produced them.

Discuss: Should headline AI benchmark scores use the standard test setup only, or publish every tested setup even when the results are less flattering?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.