NEWS · RESEARCH · #149
Asclepius: adaptive harness improves long-horizon clinical LLM agents
This arXiv paper evaluates long-horizon failures of LLM agents in the Clinical Environment Simulator (CES) and identifies three coupled failure modes—instruction-adherence drift, treatment incompleteness, and a severity-equity timeliness gap. The authors introduce Asclepius, a scaffolding that includes a self-evolving harness, an external clinical skills library, and three subagents; on held-out batches it improves critical-action correctness by 22% (p = 0.024) and, across the full ten-batch set, reports up to 25% improvement on critical actions and 13% on timeliness while preserving diagnostic accuracy.
KEY POINTS
- This arXiv paper evaluates long-horizon failures of LLM agents in the Clinical Environment Simulator (CES) and identifies three coupled failure modes—instruction-adherence drift, treatment incompleteness, and a severity-equity timeliness gap.
- The authors introduce Asclepius, a scaffolding that includes a self-evolving harness, an external clinical skills library, and three subagents; on held-out batches it improves critical-action correctness by 22% (p = 0.024) and, across the full ten-batch set, reports up to 25% improvement on critical actions and 13% on timeliness while preserving diagnostic accuracy.
- Addresses a practical safety and execution gap for LLM agents in long, continuous clinical workflows and demonstrates measurable gains in critical action correctness and timeliness.
WHY IT MATTERS
Addresses a practical safety and execution gap for LLM agents in long, continuous clinical workflows and demonstrates measurable gains in critical action correctness and timeliness.