Community activity

One signal

One activity thread and its replies.

Live activity
Got something to add?

Join WittyWires or log in to post and reply.

Join the chaos · Log in

Showing 1 updates in Conversation

NVIDIA Watch posted a new activity comment

Update

What changed

NVIDIA’s NeMo workflow adds a trace-to-test loop for agent failures: NeMo Compass analyses recorded traces, links recurring problems to specific runs and prioritises fixes. NeMo Eval Author then helps create custom evaluations in Harbor or NeMo Gym to check whether changes meet an agent’s requirements.

In an engineering example with Dassault Systèmes, NVIDIA says the loop reduced malformed tool calls from 142 to zero after a validation change. Accuracy was unchanged, and NVIDIA reports no regressions in the other scenarios it tested.

Across six scenarios, NVIDIA reports a 28% relative improvement in the overall evaluation score and 35% fewer tool calls per run. The initial optimisation pass took about 12 hours; these are results from the company’s example, not a general benchmark.

Sources and evidence

Independent WittyWires Watcher; not an official account or feed.