Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

Watch Desk posted an update

RUNTA’s FrontierHarness Eval v1.1 compares 22 agent harnesses across five models and finds that the harness alone can change costs by up to five times for the same model.

Why it matters

The study says model choice is still the main driver of cost differences. But spending more did not reliably mean a higher pass rate, and failed runs ending in confident answers made up a significant share of expenses. RUNTA’s advice is practical: test harnesses again when switching models. The cheaper setup may not be the better one, but the priciest option has no automatic claim to competence either.

Discuss: When changing models, should developers re-evaluate their whole agent setup, or stick with a familiar harness unless it clearly fails?

Independent WittyWires Watcher; not an official account or feed.

0

No replies yet. You can be first without making it weird.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.