RESEARCH · RESEARCH · #2329
MAMath-Bench: process-aware multimodal benchmark for multi-agent mathematical reasoning (arXiv:2610.26943v1)
MAMath-Bench is a process-aware benchmark for evaluating multimodal mathematical reasoning in multi-agent systems, released by Mira AI Lab (paper arXiv:2610.26943v1). It contains 1,003 problems across 11 categories and multiple modalities (text, images, audio), with fine-grained milestone annotations to measure solution correctness, reasoning-process reliability, and robustness to adversarial perturbations; evaluations show collaboration does not reliably boost final-answer accuracy, can increase intermediate-coverage while adding redundancy, and remains vulnerable to subtle adversarial reasoning errors. The dataset is available at the AgentMath GitHub repository.
KEY POINTS
- MAMath-Bench is a process-aware benchmark for evaluating multimodal mathematical reasoning in multi-agent systems, released by Mira AI Lab (paper arXiv:2610.26943v1).
- It contains 1,003 problems across 11 categories and multiple modalities (text, images, audio), with fine-grained milestone annotations to measure solution correctness, reasoning-process reliability, and robustness to adversarial perturbations; evaluations show collaboration does not reliably boost final-answer accuracy, can increase intermediate-coverage while adding redundancy, and remains vulnerable to subtle adversarial reasoning errors.
- The dataset is available at the AgentMath GitHub repository.
WHY IT MATTERS
MAMath-Bench fills an important evaluation gap by measuring not just final-answer accuracy but the reliability and adversarial robustness of multimodal multi-agent reasoning processes, which matters for building verifiable educational LLM applications.