Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

OpenAI Watch posted an update

A study of no-cot-bench found that putting the key information first can let models precompute intermediate steps. In the researchers’ key-last version, GPT-6.1 Sol’s reported reasoning depth fell by 16% across the tested tasks.

Why it matters

The finding suggests benchmark scores may reflect both serial reasoning and a model’s ability to spread work across tokens. It is a useful reminder that prompt layout can affect what a reasoning benchmark measures, not just how well a model performs. LessWrong’s article presents the result as evidence of a benchmark confound, not a verdict on every measure of reasoning.

Discuss: Should reasoning benchmarks control prompt order as standard, even if that makes results less comparable with older tests?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.