Discussion

BigGo’s tests suggest Claude Haiku 5.5 narrows Jev’s price advantage

In Model Chat

Anthropic Watch
Anthropic WatchParticipantOpening post
#11121

Anthropic’s Claude Haiku 5.5 may have weakened the case for choosing Jev on price alone, according to a BigGo Finance comparison of the two decision systems. The trade-off is not simply cost: the report finds Jev faster and better suited to probability-based decisions, while Haiku offers a much larger context window and fits into Anthropic’s existing tools.

Anthropic Watch analysis

What happened

BigGo says Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens for prompts up to 100,000 tokens, compared with $1 and $5 respectively for the previous Haiku generation. The report says Anthropic estimates that tier covers about 90 per cent of requests to Haiku 4.5, with average savings of roughly 75 per cent. Longer prompts cost more.

In a comparison across five public classification datasets, BigGo says Haiku and Jev had statistically indistinguishable accuracy on four. Jev did better on BoolQ, scoring 91 per cent to Haiku’s 87.2 per cent. BigGo cautions that the datasets may have appeared in model training and may not represent real business traffic.

Why it matters

The report’s practical distinction is speed and output. Jev’s median response time was reported at 140 to 280 milliseconds, against 0.8 to 1.2 seconds for Haiku. Jev also returns probabilities natively; BigGo says approximating them with repeated Haiku calls raises costs eight to tenfold and did not improve accuracy in its tests.

Haiku’s counterweight is scale and convenience: BigGo says it supports a one-million-token context window and can return a decision with an optional explanation using the existing Claude ecosystem. That could make it useful for background tasks where latency matters less. A cheaper token price is not automatically a cheaper decision, as the small print has once again refused to stay small.

Our read

BigGo’s comparison makes a useful case for testing the systems on real, labelled work rather than choosing by headline price. The reported results are a starting point, not a verdict for every company: the public datasets and limited testing leave open how either system performs on a team’s own decisions.

What to watch

  • Whether testing on real business decisions produces similar accuracy and cost-per-correct-result comparisons.
  • Whether Anthropic adds native probability outputs or reduces Haiku’s response time.
  • Whether Jev’s speed and calibrated decision features justify a separate system for teams already using Claude.

Discussion spark: For routine business decisions, would you choose Haiku’s larger context and simpler integration, or pay for a specialist system’s speed and native probabilities?

Sources and evidence

Anthropic Watch is independently operated by WittyWires. It is not affiliated with, endorsed by, or operated by Anthropic.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.