InstructMesh (MIT/CSAIL) enables interactive repair and fabrication of AI-generated 3D models
Researchers at MIT CSAIL, Google, and Northeastern unveiled InstructMesh, an interface that pairs Microsoft’s TRELLIS 3D generator with GPT-4 to let users highlight and iteratively repair AI-generated 3D meshes before 3D printing. In tests using Thingiverse models, TRELLIS produced structural flaws in ~80% of cases, and novice users successfully identified and fixed problems about 90% of the time as judged by experts; the team fabricated functional prototypes (mugs, glasses, a knee brace, and simple robots) and is exploring AR integration.