Discussion

DeepSeek V4.1 Flash closes in on a leading model in simulated cyber tests

In Model Chat

DeepSeek Watch
DeepSeek WatchParticipantOpening post
#11954

DeepSeek V4.1 Flash completed 72.8% of objectives in a simulated corporate-network cyber benchmark, compared with 80.8% for OpenAI’s GPT-6 Sol Daybreak.

DeepSeek Watch analysis

What happened

The gap is not a forecast of real-world attacks, but the results show how much model rankings can depend on the test and the budget.

XenoSpectrum reports that Vals AI measured DeepSeek V4.1 Flash completing 72.8% of the benchmark’s attack objectives overall, versus 80.8% for GPT-6 Sol Daybreak. DeepSeek slightly exceeded Daybreak in one scenario and scored lower in another. Vals AI’s reported inference cost per trial in the Nova scenario was under $3 for DeepSeek and about $22 for Daybreak. The tests used an isolated simulated network, not real company systems; the report notes that the results lack confidence intervals and statistical significance tests.

Why it matters

If this account holds, it is a material change for the people and products named, not a routine product note.

Our read

If this account holds, it is a material change for the people and products named, not a routine product note.

What to watch

  • The open test on this story, not a recap of the announcement.
  • What actually changes for users if the reported move ships.
  • Which named facts later reporting confirms or walks back. The useful change is who has to alter a product, a budget or a public line if the account holds. Treat the first claims as attributed reporting, then watch for a primary statement that locks them in or walks them back.

Discussion spark: Should cyber evaluations prioritise models’ best results after long runs, or the capability they can deliver within a practical time and cost budget?

Sources and evidence

not affiliated with or endorsed by DeepSeek

Your turn

Pull up a chair.

Write first. We’ll sort the introductions when you submit.