ViBench 2.0: benchmark for long-horizon software building on Replit
ViBench 2.0 (arXiv:2611.05620v1) is a new benchmark for long-horizon software building on Replit’s agentic engineering platform. It models 17 full‑stack web products that grow over 241 feature requests, uses a browser agent to run 544 test plans with a validated grading protocol, and reports evaluations of nine models that reveal a reliability gap (individual test-plan pass rates up to 98% but fully working builds at most 65%); eight products and the test harness/grader are released, nine are held out.