RESEARCH · RESEARCH · #1517
arXiv paper defines 'world editing' and releases IGMWorld plus IGMBench for Minecraft and Terraria
The paper formalizes 'world editing'—making targeted interventions in executable worlds while preserving specified properties—and introduces intervention depth as a measure of how edits couple entities, dynamics, and systems. The authors release IGMWorld and IGMBench (110 tasks, >1.1K executable state and behavioral criteria across Minecraft and Terraria) and report that top coding-agent configurations solve 78.2% of tasks under a strict task-level criterion (94.8% at the criterion level), with reliability dropping as intervention depth increases and visual consistency remaining a major weakness (joint visual pass <50%).
KEY POINTS
- The paper formalizes 'world editing'—making targeted interventions in executable worlds while preserving specified properties—and introduces intervention depth as a measure of how edits couple entities, dynamics, and systems.
- The authors release IGMWorld and IGMBench (110 tasks, >1.1K executable state and behavioral criteria across Minecraft and Terraria) and report that top coding-agent configurations solve 78.2% of tasks under a strict task-level criterion (94.8% at the criterion level), with reliability dropping as intervention depth increases and visual consistency remaining a major weakness (joint visual pass <50%).
- This introduces a new, measurable capability—world editing—with a public benchmark and baseline results, highlighting strengths and failure modes of current coding agents and enabling systematic research on editing executable interactive environments.
WHY IT MATTERS
This introduces a new, measurable capability—world editing—with a public benchmark and baseline results, highlighting strengths and failure modes of current coding agents and enabling systematic research on editing executable interactive environments.