RESEARCH · RESEARCH · #603
Detecting harmful prompts via latent activations in LLaMA-3.1-8B
The arXiv paper 'Safety Beyond the Interface' probes latent activations from LLaMA-3.1-8B and trains lightweight MLP classifiers (12.6M parameters) to detect harmful prompts. Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, the probes achieve F1 scores of 99%, 83%, and 84% respectively, claiming performance competitive with guard models that are ~1000x larger while reducing latency and compute overhead.
KEY POINTS
- The arXiv paper 'Safety Beyond the Interface' probes latent activations from LLaMA-3.1-8B and trains lightweight MLP classifiers (12.6M parameters) to detect harmful prompts.
- Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, the probes achieve F1 scores of 99%, 83%, and 84% respectively, claiming performance competitive with guard models that are ~1000x larger while reducing latency and compute overhead.
- If robust, finding harmful content in the model's latent states could enable much lower-latency, lower-cost safety checks and narrow an assurance gap left by external guard models.
WHY IT MATTERS
If robust, finding harmful content in the model's latent states could enable much lower-latency, lower-cost safety checks and narrow an assurance gap left by external guard models.