RESEARCH · RESEARCH · #2544
PersonaScienceBench: benchmark for scientific agents using diverse simulated researcher personas (arXiv:2610.32106v1)
PersonaScienceBench is a new evaluation benchmark (arXiv:2610.32106v1) containing 200 tasks drawn from five prior benchmarks across four scientific domains. It pairs tasks with simulated researcher personas that give contextual preferences and conversational feedback; agents therefore must adapt through interaction. The paper reports results across eighteen model configurations and targeted analyses using GPT‑6 Astra, showing that task scores, factual recovery, and interaction behaviors can diverge — agents may behave differently with different researcher profiles even when final scores are similar.
KEY POINTS
- PersonaScienceBench is a new evaluation benchmark (arXiv:2610.32106v1) containing 200 tasks drawn from five prior benchmarks across four scientific domains.
- It pairs tasks with simulated researcher personas that give contextual preferences and conversational feedback; agents therefore must adapt through interaction.
- The paper reports results across eighteen model configurations and targeted analyses using GPT‑6 Astra, showing that task scores, factual recovery, and interaction behaviors can diverge — agents may behave differently with different researcher profiles even when final scores are similar.
WHY IT MATTERS
This benchmark measures how well scientific agents adapt to diverse collaborator preferences via conversation, exposing behavior differences that task scores alone can miss.