RESEARCH · RESEARCH · #2866
LLMON post-training teaches explicit instruction/data separation and cuts prompt-injection success
The paper introduces LLMON (LLM Object Notation), a post-training supervision format that enforces explicit identifier-based execution by separating instructions from data. Using a 4.6B-token structured corpus (including RAG and tool-calling data with distractor-infused variants), the authors fine-tune Granite-4.1 3B and 8B models via supervised fine-tuning followed by reinforcement learning and report that LLMON supervision reduces indirect prompt-injection attack success rates from 53–100% down to 0.02–5.6% while preserving general capabilities; they also propose attention-based interpretability metrics that reveal how attention patterns change under structured inputs.
KEY POINTS
- The paper introduces LLMON (LLM Object Notation), a post-training supervision format that enforces explicit identifier-based execution by separating instructions from data.
- Using a 4.6B-token structured corpus (including RAG and tool-calling data with distractor-infused variants), the authors fine-tune Granite-4.1 3B and 8B models via supervised fine-tuning followed by reinforcement learning and report that LLMON supervision reduces indirect prompt-injection attack success rates from 53–100% down to 0.02–5.6% while preserving general capabilities; they also propose attention-based interpretability metrics that reveal how attention patterns change under structured inputs.
- Demonstrates that explicit execution semantics can be taught via standard post-training to markedly reduce prompt-injection vulnerabilities without architecture changes, improving model safety and control.
WHY IT MATTERS
Demonstrates that explicit execution semantics can be taught via standard post-training to markedly reduce prompt-injection vulnerabilities without architecture changes, improving model safety and control.