NVIDIA researcher says synthetic training data lifted Nemotron’s search score to 30%

NVIDIA research scientist Dhruv Nathawani says a synthetic-data workflow raised Nemotron Nano’s BrowseComp score from 0% to 30%. The workshop also shows how the data is made, and why checking it remains the hard part.

NVIDIA Watch analysis

What happened

Nathawani’s workshop, covered by BigGo Finance, walks through NVIDIA’s open-source Data Designer framework. It builds datasets from columns arranged into a workflow, using real-world seeds and generated examples. For agent search, the demo starts with entities from Wikidata, builds questions designed to require web searches, then records the agent’s search queries, results and answers.

Nathawani says the search dataset moved Nemotron Nano from 0% to 30% on BrowseComp. He also reports an improvement from 26% to 41% on the Bird text-to-SQL benchmark. The results are his account of NVIDIA’s work, not an independent benchmark replication. In one example, filtering reduced 50,000 starting seeds to 7,000 high-quality training records.

Why it matters

The practical point is that teaching an AI agent to use tools may depend as much on the training examples as on the model. Data Designer lets developers create and inspect those examples, including full tool-use trajectories, rather than training only on an agent’s final answer.

The workshop also gives a useful warning: a plausible-looking dataset can be wrong. Nathawani describes a Wikidata seed that had gone stale, producing an outdated answer that a simple verifier accepted. More generated data is not automatically better data; the filtering and checking are doing important work.

Our read

This is a concrete account of how synthetic data can improve a specific agent capability, with benchmark numbers readers can interrogate rather than a vague promise of smarter AI. The most valuable part may be the less glamorous one: inspect samples, iterate, and check whether your verification method is itself reliable. If you use generated training data, treat its quality controls as part of the system, not the tidy-up afterwards.

What to watch

• Whether NVIDIA publishes enough detail for others to reproduce the BrowseComp and Bird results.
• How well the generated examples hold up against current, changing facts.
• Whether Data Designer’s filtering and evaluation methods transfer to other agent tasks.

Discussion spark: When synthetic data improves an agent benchmark, what evidence should developers publish before others trust the result?

Independent WittyWires tracker for public updates about NVIDIA. Not affiliated with or endorsed by NVIDIA; this is not an official account.