RESEARCH · RESEARCH · #1390
InstructMesh (MIT/CSAIL) enables interactive repair and fabrication of AI-generated 3D models
Researchers at MIT CSAIL, Google, and Northeastern unveiled InstructMesh, an interface that pairs Microsoft’s TRELLIS 3D generator with GPT-4 to let users highlight and iteratively repair AI-generated 3D meshes before 3D printing. In tests using Thingiverse models, TRELLIS produced structural flaws in ~80% of cases, and novice users successfully identified and fixed problems about 90% of the time as judged by experts; the team fabricated functional prototypes (mugs, glasses, a knee brace, and simple robots) and is exploring AR integration.
KEY POINTS
- Researchers at MIT CSAIL, Google, and Northeastern unveiled InstructMesh, an interface that pairs Microsoft’s TRELLIS 3D generator with GPT-4 to let users highlight and iteratively repair AI-generated 3D meshes before 3D printing.
- In tests using Thingiverse models, TRELLIS produced structural flaws in ~80% of cases, and novice users successfully identified and fixed problems about 90% of the time as judged by experts; the team fabricated functional prototypes (mugs, glasses, a knee brace, and simple robots) and is exploring AR integration.
- This matters because it combines generative 3D modeling and language-model-guided editing to lower the expertise barrier for producing physically functional, 3D-printable objects, which could speed up consumer customization and robotic enclosure design.
WHY IT MATTERS
This matters because it combines generative 3D modeling and language-model-guided editing to lower the expertise barrier for producing physically functional, 3D-printable objects, which could speed up consumer customization and robotic enclosure design.