Plotly says its comparison of 57 language models found top performers answering all 43 questions in its data-analysis benchmark, up from 20 two years earlier. The company also says execution costs fell significantly, though it gives no figure in the supplied account.
Watch Desk analysis
What happened
In a report published on 12 November, Plotly describes evaluating models released between November 2024 and November 2026 with its DataAnalyticsBench exam. The top result improved from 20 correct answers out of 43 to all 43. Plotly also reports a significant drop in execution costs over that period.
That is a result on one benchmark, not a guarantee that models can handle every spreadsheet, chart or awkward question a workplace can throw at them. The report’s supplied summary does not quantify the cost reduction.
Why it matters
Data analysis is a practical test of whether a model can do more than produce fluent prose: it must get answers right on a defined set of questions, and do so at a cost people can use. Plotly’s figures point to progress on both fronts, although the missing cost detail makes that part of the story harder to measure.
For teams weighing up AI-assisted analysis, the useful takeaway is to ask for performance on their own tasks and the cost of getting those answers, rather than treating a benchmark clean sweep as a universal warranty. A perfect score is impressive; it is also 43 questions wide.
Our read
This is a meaningful snapshot of improving AI data-analysis capability, with the welcome possibility that better performance is becoming cheaper to run. Plotly makes the benchmark and its scope clear, but readers need the underlying results and a concrete cost comparison to judge how much the gains travel beyond this test.
What to watch
- The detailed benchmark results and how models performed across different question types.
- A quantified comparison of execution costs across the two-year period.
- Whether other evaluations find similar progress on real-world data-analysis work.
Discussion spark: When choosing an AI tool for data analysis, would you put more weight on benchmark accuracy or on how much it costs to get a reliable answer?
Sources and evidence
- Two Years of LLM Progress on Data Analysis (12 November 2026, 00:00 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.