RESEARCH · RESEARCH · #2094
Paper finds fragility in skill-policy co-evolution and introduces the CODA drift audit
The arXiv paper 'Evolving Together, Failing Apart' analyzes the training paradigm where an agent's policy and its skill bank co-evolve, showing theoretically and empirically that cooperative acceptance links a policy's competence to the specific bank it trained with and makes it fragile when skills drift post-deployment. The authors propose CODA (CO-evolution Drift Audit), a deployment-oriented protocol that simulates skill drift and adds metrics — worst-case success under drift, a consistency rate for meaning-preserving edits, and a resistance rate to corrupted skills — and report that across four co-evolution methods and five benchmarks, worst-case success is 5–13 points below peak, erasing 52–93% of delivered-bank gains over a no-skill baseline.
KEY POINTS
- The arXiv paper 'Evolving Together, Failing Apart' analyzes the training paradigm where an agent's policy and its skill bank co-evolve, showing theoretically and empirically that cooperative acceptance links a policy's competence to the specific bank it trained with and makes it fragile when skills drift post-deployment.
- The authors propose CODA (CO-evolution Drift Audit), a deployment-oriented protocol that simulates skill drift and adds metrics — worst-case success under drift, a consistency rate for meaning-preserving edits, and a resistance rate to corrupted skills — and report that across four co-evolution methods and five benchmarks, worst-case success is 5–13 points below peak, erasing 52–93% of delivered-bank gains over a no-skill baseline.
- The paper identifies a systemic evaluation blind spot and provides CODA as a practical standard to measure whether co-evolving agents remain reliable when their skill banks change after deployment.
WHY IT MATTERS
The paper identifies a systemic evaluation blind spot and provides CODA as a practical standard to measure whether co-evolving agents remain reliable when their skill banks change after deployment.