Synthetic charts lift Nemotron’s tick-count accuracy from 17% to 78%

A Thoughtworks researcher reports that a short fine-tuning run sharply improved NVIDIA’s Nemotron 3.5 Super VL at counting labelled ticks across chart axes, raising its score on 224 CharXiv questions from 16.96% to 78.13%. The result is a concrete example of synthetic training data helping a model combine visual details it could already read.

Watch Desk analysis

What happened

Gustavo A. Lujan’s report describes training a LoRA adapter for 3.3 hours on 24,000 generated charts. None of the CharXiv evaluation figures were used for training. On a separate set of 300 generated charts, accuracy rose from 18.33% to 99.33%.

The task required the model to locate chart axes, read their tick labels and add them up. The original checkpoint handled the component skills much better than the combined task. Lujan also reports that the model’s built-in multi-token prediction head increased single-request output speed by a factor of 1.56 on the original checkpoint.

Why it matters

The useful lesson is not that a model has become a universal chart-reading expert. It is that teams may be able to target a specific failure in a vision-language model with generated examples, then test the result on real benchmark figures that were kept out of training. Thoughtworks recommends the same pattern for customer archives: generate charts shaped around the question, then measure performance on the customer’s own charts.

The numbers come from one narrowly defined chart-counting task, not a broad measure of visual reasoning. The experiments also used an early-access checkpoint, and the report says the results have not been re-measured on the later general-availability checkpoints.

Our read

This is a notably practical fine-tuning result: the model could read individual labels, but struggled to assemble them into a total. Synthetic data helped close that gap in this test. Teams considering a similar approach should take the recipe seriously, but treat its score as a task-specific result, not a transferable guarantee.

What to watch

• Whether the reported gains hold on the general-availability checkpoints.
• Whether the approach works on customer charts with different layouts and styles.
• Whether other teams reproduce the result on comparable tasks.

Discussion spark: Would you trust synthetic examples to improve a model for your own chart archive, or would you require a separate real-world evaluation before deployment?

Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.