PARTNERSHIP · COMPANIES · #1725
Goodfire launches internal-activation monitors on Baseten to catch rogue AI agents
Goodfire launched cheaper “inside-out” monitors that read a model’s internal activations (probes) in real time and are available to Baseten customers; in tests on Kimi K3 the probes flagged 94% of malicious hacking sessions, sent 8.7% of benign sessions for review, and monitoring ~1,500 sessions cost about $51 vs $233–$10,000 for alternative checkers. Baseten’s Base Labs had announced a safety partnership with Goodfire and Hugging Face last month, and Goodfire cites similar research from Google DeepMind informing probe-based detection in Gemini.
KEY POINTS
- Goodfire launched cheaper “inside-out” monitors that read a model’s internal activations (probes) in real time and are available to Baseten customers; in tests on Kimi K3 the probes flagged 94% of malicious hacking sessions, sent 8.7% of benign sessions for review, and monitoring ~1,500 sessions cost about $51 vs $233–$10,000 for alternative checkers.
- Baseten’s Base Labs had announced a safety partnership with Goodfire and Hugging Face last month, and Goodfire cites similar research from Google DeepMind informing probe-based detection in Gemini.
- This matters because probe-based internal monitoring promises much cheaper, lower-latency inference-time guardrails for open models and agents, addressing recent agent escape and reward-hacking incidents.
WHY IT MATTERS
This matters because probe-based internal monitoring promises much cheaper, lower-latency inference-time guardrails for open models and agents, addressing recent agent escape and reward-hacking incidents.