RESEARCH · RESEARCH · #1317
ChartRevise: dataset and protocol for exact program-grounded chart editing
ChartRevise is a structured dataset and a reference-free evaluation protocol for exact program-grounded chart editing, constructed using the grammar of graphics to cover 344 edit types across 20 chart types and three plotting libraries, with 92,438 records. The protocol measures atomic requirement completion, flags gratuitous changes and missed coupled updates, and combines these checks with execution and rendering; fine-tuning on the dataset yielded +16% mean requirement recall and +22% mean exact-edit rate across five models and four external benchmarks.
KEY POINTS
- ChartRevise is a structured dataset and a reference-free evaluation protocol for exact program-grounded chart editing, constructed using the grammar of graphics to cover 344 edit types across 20 chart types and three plotting libraries, with 92,438 records.
- The protocol measures atomic requirement completion, flags gratuitous changes and missed coupled updates, and combines these checks with execution and rendering; fine-tuning on the dataset yielded +16% mean requirement recall and +22% mean exact-edit rate across five models and four external benchmarks.
- It provides a large, systematic benchmark and a fine-grained, automated evaluation that distinguishes requested edits from coupled or gratuitous code changes, improving training and measurement for models that edit chart-generating code.
WHY IT MATTERS
It provides a large, systematic benchmark and a fine-grained, automated evaluation that distinguishes requested edits from coupled or gratuitous code changes, improving training and measurement for models that edit chart-generating code.