Measuring Where Intent Gets Lost in AI Synthesis of Smart Contracts (arXiv:2610.37830v1)
This new arXiv paper builds an execution-based evaluation to measure how often AI-generated smart contracts behave differently from their intended originals by replaying the same transactions on both and measuring agreement. The authors find common graders and automated fixes miss obvious mistakes, show that checking only included functions conceals many differences, propose an 'elicitation-first' workflow where the agent asks a short requirements checklist before coding, and release their tools, 600 tasks, and all results; asking before generation recovered much more intended behavior across three models than post-hoc review or automated patches.