AI engineers may be shifting from writing individual features to building systems in which agents inspect telemetry, find problems and help make fixes. Arize AI co-founder Aparna Dhinakaran describes that approach through Signal, the company’s always-on agent for analysing software traces.
Watch Desk analysis
What happened
At AI Engineer New York, Dhinakaran outlined how she sees agents changing the build, evaluation and monitoring cycle. Rather than relying on people to comb through dashboards, teams can set agents to examine telemetry and identify patterns, according to an account of her talk by BigGo Finance.
Dhinakaran described Signal as an agent that can wake on a schedule, a message such as a user complaint, or a metric threshold. It gathers relevant trace data and other context in a sandbox; if given repository access, it can propose a fix or open a pull request. The system saves findings outside the sandbox so later runs can build on them, while tearing down the sandbox, credentials and repository access when a run ends.
The account says Signal supports Claude Code, Cursor and Codex, and can use different models within the Anthropic family. Dhinakaran presented that flexibility as a way to compare quality and cost rather than assume a more powerful model is always worth the bill.
Why it matters
This is a concrete picture of agentic software work: agents do not simply generate code, they can be connected to the monitoring and evaluation processes that determine what needs fixing next. That could make software teams faster at spotting issues, but it also makes the quality of their tests and automated judgements central to the whole loop.
Dhinakaran says Arize checks its evaluators against human judgement using curated datasets. That is a useful practice to watch, not proof that the system’s judgements are reliable: the account gives no agreement rates or error bounds.
Our read
The interesting shift is from “let an agent write code” to “design the system that tells agents what to do, checks their work and learns from the result”. That is a more demanding job than adding a chatbot to a workflow, and rather less likely to be solved by a prompt with a jaunty name.
For teams trying this, the transferable lesson is to measure whether an evaluator agrees with people before trusting it to grade production work. Treat the agent loop as an engineered system, not a shortcut with a dashboard.
What to watch
- Whether Signal publishes measured results for its evaluators and generated fixes.
- How model and coding-tool choices affect the quality and cost of each run.
- Whether teams can keep the loop useful when their production data is messy and incomplete.
Discussion spark: If agents are reading telemetry and proposing fixes, what should teams require before letting their judgements shape production software?
Sources and evidence
- Arize AI's Aparna Dhinakaran: AI Engineers Now Build the Loop, Not the Feature – BigGo Finance (25 October 2026, 21:08 UTC)
Watch Desk is operated by WittyWires as an independent cross-cutting AI news tracker. It does not speak for the organisations or people it covers.