RESEARCH · RESEARCH · #2623
BenchBench-Cloning benchmark evaluates LLMs on end-to-end molecular cloning planning
Researchers publish BenchBench-Cloning (arXiv:2610.33355v1), a 65-task benchmark for large language models focused on end-to-end molecular cloning: 50 protocol-development tasks evaluated by execution in a cloning simulator and 15 construct-design tasks evaluated with deterministic checks and rubric-based LLM-as-a-judge reviews. Across eight systems tested with repeated attempts, GPT-6 Astra scored highest at 30.6% (others 8.6–22.2%), and the benchmark remained unsaturated under best-of-ten sampling, highlighting ongoing challenges for experimental planning.
KEY POINTS
- Researchers publish BenchBench-Cloning (arXiv:2610.33355v1), a 65-task benchmark for large language models focused on end-to-end molecular cloning: 50 protocol-development tasks evaluated by execution in a cloning simulator and 15 construct-design tasks evaluated with deterministic checks and rubric-based LLM-as-a-judge reviews.
- Across eight systems tested with repeated attempts, GPT-6 Astra scored highest at 30.6% (others 8.6–22.2%), and the benchmark remained unsaturated under best-of-ten sampling, highlighting ongoing challenges for experimental planning.
- BenchBench-Cloning matters because it measures end-to-end experimental planning — a harder, more realistic task than isolated knowledge questions — and shows even frontier LLMs struggle, guiding future research priorities.
WHY IT MATTERS
BenchBench-Cloning matters because it measures end-to-end experimental planning — a harder, more realistic task than isolated knowledge questions — and shows even frontier LLMs struggle, guiding future research priorities.