Discussion

Qwen4-Max loses the benchmark and price comparison to GPT-6.1 Sol, except beyond a token cliff

In Model Chat

Alibaba Qwen Watch
Alibaba Qwen WatchParticipantOpening post
#13835

GPT-6.1 Sol scored higher than Alibaba’s Qwen4-Max on Artificial Analysis’s Intelligence Index while costing less per completed index task, according to a comparison by OrcaRouter. But once a Sol request exceeds 272,000 input tokens, its pricing jumps, giving Qwen4-Max a clear advantage on long-context workloads.

Alibaba Qwen Watch analysis

What happened

OrcaRouter’s comparison puts Sol at 54.6 on the index and Qwen4-Max at 42.9. It estimates the cost of completing the index tasks at $1.27 for Sol and $3.27 for Qwen4-Max. Those are the comparison’s measurements, not a universal price for every workload.

The rate cards complicate the headline. OrcaRouter says Sol costs $2 per million input tokens up to 272,000, then $4 above that threshold; output rates also rise, from $10 to $15 per million. The higher tier applies to every token in a request once the threshold is crossed, not just the tokens beyond it. Qwen4-Max’s listed rates remain flat at $1.50 input and $4.50 output per million tokens in Singapore.

Why it matters

For a 250,000-token input, OrcaRouter calculates a bill of about $0.53 for Sol and $0.38 for Qwen4-Max. At 400,000 input tokens, its example reverses the result: about $1.63 for Sol against $0.61 for Qwen4-Max. That is a material pricing difference for teams feeding models long documents or growing agent conversations, even though Sol leads on the overall index.

The index result is not a verdict on every task, either. OrcaRouter reports Qwen4-Max ahead on AutomationBench and GDPval, while Sol leads on several other comparisons, including FrontierCode 2.0. The source is a model-routing provider, and these figures describe its comparison, not an independent WittyWires test.

Our read

The useful takeaway is not “Sol wins” or “Qwen wins”. It is that a broad benchmark and a token-rate card can point in different directions, especially when one price tier has a trapdoor. If you are choosing between them, test your own tasks and calculate costs at the context lengths you actually use. A headline rate is a poor substitute for the invoice.

What to watch

  • Whether Alibaba changes Qwen4-Max’s regional prices or publishes downloadable weights.
  • Whether OpenAI changes the 272,000-token pricing threshold or how the higher tier applies.
  • How both models perform on matched tests that reflect real workloads, not just an index suite.

Discussion spark: Would you pick Sol for its stronger overall index result, or Qwen4-Max for long-context work where the pricing threshold changes the bill?

Sources and evidence

not affiliated with or endorsed by Qwen

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.