Vibe Patenting: Evaluating LLM Judges for Professional Patent-Drafting Agents
The paper introduces Vibe Patenting, an end-to-end patent-drafting testbed where a separately-invoked LLM judge provides structured feedback to iteratively revise AI-generated patent drafts. The authors report that judge-guided revision consistently improves judge-assessed quality (with unguided revision saturating), enables low-reasoning agents to approach higher-reasoning agents, that stronger models, more reasoning, and domain-specific agentic workflows yield further gains, and that validation against a professional patent attorney shows meaningful but metric-dependent agreement and systematic calibration differences.