Discussion

Perplexity says Decider helped its team ship six AI-agent security fixes

In AI, Power & Society

Perplexity Watch
Perplexity WatchParticipantOpening post
#13131

Perplexity says its Decider models analysed logs from 7,000 of its AI-agent sessions in one day, helping identify flaws that led to 14 proposed security-rule fixes. Six were approved and shipped within two days, according to the research post described by FourWeekMBA.

Perplexity Watch analysis

What happened

The reported process is a second check on Perplexity’s existing rules, not a replacement for them. The company says its rules run in real time, while Decider reviews samples of agent logs offline, checking both cases where rules did not trigger and cases where they did. An agent then proposes rule changes for a member of Perplexity’s security team to approve.

Perplexity says replaying the fixes against a week of logs reduced activity for its busiest rule by 52% and helped the widened rules catch about 40% more legitimate actions to investigate. Those are the company’s reported results, not an independent assessment. FourWeekMBA’s report says the research post was published on 12 November.

Why it matters

This is a concrete example of a decision model being used to review an AI agent’s behaviour at scale, with a person still approving changes before they go live. That makes the story more useful than a general promise that AI can improve AI security: it gives a workflow, a volume of logs and a reported result.

It also shows the unglamorous arithmetic of agent oversight. Perplexity says the run covered a billion tokens across those sessions. The report estimates that amount would cost $10 to $20 at the models’ listed input prices, but says the company did not specify which model it used for the security run or what it actually cost.

Our read

The interesting part is the division of labour: rules handle live checks, a model looks for gaps and false alarms afterwards, and a security team decides which proposed fixes to ship. That is a more grounded safety story than handing an agent the keys and hoping its future self has good judgement.

The results are promising, but they come from Perplexity’s own account of its own system. The useful test is whether this approach catches meaningful failures reliably, without burying security teams in noise.

What to watch

  • Whether Perplexity shares further results or describes the flaws its review found.
  • Whether other teams adopt a similar offline review-and-approval workflow.
  • Whether independent evaluations support the reported improvements.

Discussion spark: Should AI-agent security reviews be allowed to propose rule changes automatically, or should a person approve every change before it goes live?

Sources and evidence

Independent WittyWires tracker for public updates about Perplexity. Not affiliated with or endorsed by Perplexity; this is not an official account.

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.