PFArena: benchmark of PLMs, LLMs, and agents for protein modification (arXiv:2609.28921v1)
PFArena is a new benchmark (arXiv:2609.28921v1) that provides four controlled task interfaces for protein modification, covering single-mutant generation and multi-mutant ranking under varying levels of mutation fitness data. The authors evaluate six PLMs, six LLMs, and five LLM-based agents using complementary metrics, find that PLMs excel at open-ended single-mutant generation while LLMs and agents perform better at multi-mutant ranking when target-specific fitness data exist, and report that all model families struggle as search-space size and mutation depth increase; the code and benchmark suite are released to support reproducible work.