RESEARCH · RESEARCH · #2318
PROVISO: provenance-informed controller for secure LLM agents (arXiv:2610.27115v1)
PROVISO is a controller that decides whether to commit an LLM agent's proposed action by encoding trusted observations, external tool outputs, the proposed action, and an alternative into provenance-typed memory and selecting the action with the highest calibrated value learned from outcomes. Evaluated on four agent security tasks, PROVISO outperforms baselines — improving F1 by 13.84 points on SkillsBench and 7.07 points on ToolSafe, and achieving 88–99 F1 on three additional skill detection benchmarks.
KEY POINTS
- PROVISO is a controller that decides whether to commit an LLM agent's proposed action by encoding trusted observations, external tool outputs, the proposed action, and an alternative into provenance-typed memory and selecting the action with the highest calibrated value learned from outcomes.
- Evaluated on four agent security tasks, PROVISO outperforms baselines — improving F1 by 13.84 points on SkillsBench and 7.07 points on ToolSafe, and achieving 88–99 F1 on three additional skill detection benchmarks.
- Tracking provenance of inputs and comparing proposed actions to alternatives lets PROVISO reduce harmful actions while preserving utility, addressing a key safety-utility trade-off for tool-using LLM agents.
WHY IT MATTERS
Tracking provenance of inputs and comparing proposed actions to alternatives lets PROVISO reduce harmful actions while preserving utility, addressing a key safety-utility trade-off for tool-using LLM agents.