RESEARCH · RESEARCH · #3239
Vals study finds open-weight DeepSeek nears OpenAI Daybreak in simulated hacking tasks
Vals evaluated multiple AI models on simulated multi-step corporate network intrusion scenarios and found Chinese open-weight model DeepSeek completed about 73% of objectives versus OpenAI’s Daybreak at 81%, with DeepSeek much cheaper to run; the firm warns open models are beginning to match formerly gated frontier models. The tests used two mock corporate-network scenarios and measured task completion rates and token costs (Vals reported $2–$3 for DeepSeek vs $17–$46 for GPT-6-Sol).
KEY POINTS
- Vals evaluated multiple AI models on simulated multi-step corporate network intrusion scenarios and found Chinese open-weight model DeepSeek completed about 73% of objectives versus OpenAI’s Daybreak at 81%, with DeepSeek much cheaper to run; the firm warns open models are beginning to match formerly gated frontier models.
- The tests used two mock corporate-network scenarios and measured task completion rates and token costs (Vals reported $2–$3 for DeepSeek vs $17–$46 for GPT-6-Sol).
- If open-weight models can match closed frontier models in cyber capabilities, gated-release strategies and defender assumptions about limited attacker access may no longer hold.
WHY IT MATTERS
If open-weight models can match closed frontier models in cyber capabilities, gated-release strategies and defender assumptions about limited attacker access may no longer hold.