NEWS · RESEARCH · #362
CADWorld: benchmark for long-horizon mechanical CAD workflows in FreeCAD
CADWorld is a new benchmark (arXiv:2609.16251v1) for long-horizon computer-aided design in FreeCAD, comprising 200 tasks across 11 mechanical-CAD workflow categories (sketching, part modeling, assembly, CAM, FEM, measurement, mesh processing, technical drawing, etc.). Agents interact via screenshots and GUI actions and are evaluated by executable checks on saved FreeCAD artifacts; across seven agents the top score is 17.5% success versus an 87.0% expert pass rate, highlighting major gaps in producing persistent, verifiable engineering artifacts.
KEY POINTS
- CADWorld is a new benchmark (arXiv:2609.16251v1) for long-horizon computer-aided design in FreeCAD, comprising 200 tasks across 11 mechanical-CAD workflow categories (sketching, part modeling, assembly, CAM, FEM, measurement, mesh processing, technical drawing, etc.).
- Agents interact via screenshots and GUI actions and are evaluated by executable checks on saved FreeCAD artifacts; across seven agents the top score is 17.5% success versus an 87.0% expert pass rate, highlighting major gaps in producing persistent, verifiable engineering artifacts.
- CADWorld matters because it evaluates agents on producing persistent, structured engineering artifacts and reveals a substantial gap between general GUI competence and reliable execution of end-to-end CAD workflows.
WHY IT MATTERS
CADWorld matters because it evaluates agents on producing persistent, structured engineering artifacts and reveals a substantial gap between general GUI competence and reliable execution of end-to-end CAD workflows.