RESEARCH · RESEARCH · #934
Policy-as-Skill: governed LLM decision-support runtime (arXiv:2609.27087v1)
The paper introduces Policy-as-Skill (PaS), a modular runtime that packages evidence validation, review routing, version control, and auditability as executable, versioned policy capabilities. Evaluated with 13 methods on 600 development tasks using a fixed Gemma4 backend, PaS+Audit reported 53.8% exact accuracy, macro-F1 0.346, review F1 0.854, citation precision 1.000, policy-reference recall 0.984, and audit completeness 1.000, and the authors report that deterministic control can raise aggregate accuracy to 61.2% though results are task-dependent; PaS+Audit outperforms LLM+RAG on most governance and review metrics.
KEY POINTS
- The paper introduces Policy-as-Skill (PaS), a modular runtime that packages evidence validation, review routing, version control, and auditability as executable, versioned policy capabilities.
- Evaluated with 13 methods on 600 development tasks using a fixed Gemma4 backend, PaS+Audit reported 53.8% exact accuracy, macro-F1 0.346, review F1 0.854, citation precision 1.000, policy-reference recall 0.984, and audit completeness 1.000, and the authors report that deterministic control can raise aggregate accuracy to 61.2% though results are task-dependent; PaS+Audit outperforms LLM+RAG on most governance and review metrics.
- It matters because PaS packages governance features (evidence, review routing, versioning, audit) into an executable runtime and reports improved governance/review metrics versus LLM+RAG, showing a practical path for more auditable LLM decision support.
WHY IT MATTERS
It matters because PaS packages governance features (evidence, review routing, versioning, audit) into an executable runtime and reports improved governance/review metrics versus LLM+RAG, showing a practical path for more auditable LLM decision support.