Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

NEWS · MODELS · #3259

Vals benchmark finds open-weight Deepseek close to OpenAI's GPT-6 Sol on autonomous cyber attacks

Vals AI's CyberOffense benchmark simulated realistic corporate networks and ran solo AI agents starting from a public web address; OpenAI's GPT-6 Sol (Daybreak) achieved 80.8% of objectives, while the open-weight Deepseek V4.1 Flash 1017 reached 72.8% at much lower cost, with longer runtimes proving critical, Vals reports. The findings echo a UK AISI analysis and Anthropic tests showing narrower gaps between closed and open models, and Vals said the same models could be used defensively to find vulnerabilities.

KEY POINTS

  1. Vals AI's CyberOffense benchmark simulated realistic corporate networks and ran solo AI agents starting from a public web address; OpenAI's GPT-6 Sol (Daybreak) achieved 80.8% of objectives, while the open-weight Deepseek V4.1 Flash 1017 reached 72.8% at much lower cost, with longer runtimes proving critical, Vals reports.
  2. The findings echo a UK AISI analysis and Anthropic tests showing narrower gaps between closed and open models, and Vals said the same models could be used defensively to find vulnerabilities.
  3. Shows that open-weight models are approaching closed frontier models in autonomous cyber-offense capability at much lower cost, raising access, defense and policy risks and opportunities.

WHY IT MATTERS

Shows that open-weight models are approaching closed frontier models in autonomous cyber-offense capability at much lower cost, raising access, defense and policy risks and opportunities.

SOURCES & TIMELINE

1