NEWS · RESEARCH · #138
Identity Is More Than Recall — PAI-Bench: a benchmark for persistent identity in deployed AI agents (arXiv)
An arXiv paper introduces PAI-Bench, a provider-neutral benchmark and evaluation protocol that separates factual recall from identity expression and behavioral enactment for deployed AI agents. The study runs two frozen campaigns over 16 synthetic profiles and reports results (1,536 retained responses) showing prompt- and startup-cue-dependent differences in identity-component presence, as well as evaluator sensitivity differences between tested deployments (e.g., Claude vs. Astra); the authors note single-sample conditions and post-hoc follow-ups.
KEY POINTS
- An arXiv paper introduces PAI-Bench, a provider-neutral benchmark and evaluation protocol that separates factual recall from identity expression and behavioral enactment for deployed AI agents.
- The study runs two frozen campaigns over 16 synthetic profiles and reports results (1,536 retained responses) showing prompt- and startup-cue-dependent differences in identity-component presence, as well as evaluator sensitivity differences between tested deployments (e.g., Claude vs.
- Astra); the authors note single-sample conditions and post-hoc follow-ups.
WHY IT MATTERS
Distinguishing recall, expression, and enactment of identity addresses a practical trust-and-safety gap for deployed agents and provides a reproducible protocol for measuring multiple facets of identity fidelity.