Kimi and Grok tie on OrcaRouter’s analytics test, but Grok’s run costs less

Kimi K3.1 Pro and Grok 4.7 both scored 42 out of 43 on an analytics test run by OrcaRouter, which says the Kimi run cost $13.47 against $7.85 for Grok. The comparison offers a useful, if narrow, look at how model choice can change the bill without changing the score.

Watch Desk analysis

What happened

OrcaRouter says its private test used 43 questions about a synthetic wind-farm data set, including checks for ambiguous requests, data-quality problems and patterns models were not asked to find. Both models tied for third among 34 tested. The Kimi run found all five planted patterns and caught three data-quality problems; Grok also scored 42.

The reported results come from OrcaRouter’s own tests, not a published independent benchmark. Other comparisons in the article use different tests and setups, so their scores are not clean head-to-head measurements.

Why it matters

A tie on this test did not mean a tie on cost: OrcaRouter reports that Grok’s run was $5.62 cheaper. The result is a reminder that model performance and the price of getting that performance are separate questions, particularly for teams choosing an API for data analysis.

It is not a universal verdict. The test is one private evaluation, and the article describes other tasks where the models’ relative strengths differ. Buyers should treat it as a reason to test their own workload, not as a league table with the hard work already done.

Our read

The most useful number here is not 42. It is the price attached to the same score, under one test setup. That makes this a worthwhile comparison, with the usual small print that private tests are not a substitute for trying the models on your own work.

What to watch

• Whether OrcaRouter publishes the questions, scoring method or enough detail for others to reproduce the result.
• Whether the cost difference persists on workloads outside this synthetic data set.
• How the models compare on a broader set of independent, like-for-like tests.

Discussion spark: If two models score the same on a task, should the lower-cost option be the default, or do you need broader testing before switching?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.