MathVis benchmark (arXiv:2610.12954v1) evaluates LLM diagram generation for math
MathVis is a new benchmark (arXiv:2610.12954v1) of 512 tasks that tests large language models' ability to visualize mathematical problems and produce executable TikZ/SVG diagrams, providing reference diagrams, a Gold intermediate representation, atomic rubrics, and error codes. In initial evaluations GPT-6-Astra, Claude Fable 5, and Gemini 3.1 Pro Preview scored pass@1 of 31.25%, 26.56%, and 22.85% respectively, and the authors report LLM outputs are often formal-language-heavy and visually confusing rather than demonstrating explicit diagram-mediated reasoning.