RESEARCH · RESEARCH · #2993
ArXiv paper releases ProClaim and ProClaim-eval for consensus scientific claim verification
The paper (arXiv:2610.40906v1) introduces ProClaim, an agent instrument that persists a structured evidence state outside a model's context, and ProClaim-eval, a 718-claim dataset with SUPPORT/REFUTE gold labels from five expert-curated sources across molecular biology, nuclear physics, and astronomy. The authors find that maintaining an external evidence state and using a trained stopping judge improves consensus claim verification compared with in-context accumulation and fixed-round stopping, and that a separately trained judge matches or exceeds performance of some off-the-shelf Claude judges on parts of the dataset.
KEY POINTS
- The paper (arXiv:2610.40906v1) introduces ProClaim, an agent instrument that persists a structured evidence state outside a model's context, and ProClaim-eval, a 718-claim dataset with SUPPORT/REFUTE gold labels from five expert-curated sources across molecular biology, nuclear physics, and astronomy.
- The authors find that maintaining an external evidence state and using a trained stopping judge improves consensus claim verification compared with in-context accumulation and fixed-round stopping, and that a separately trained judge matches or exceeds performance of some off-the-shelf Claude judges on parts of the dataset.
- This matters because it provides a reproducible instrument and dataset showing that recursive, external evidence states and learned stopping criteria can measurably improve LLM-based verification of scientific consensus.
WHY IT MATTERS
This matters because it provides a reproducible instrument and dataset showing that recursive, external evidence states and learned stopping criteria can measurably improve LLM-based verification of scientific consensus.