Decision language model Jev rivals many LLM judges for clinical AI evaluation while being far cheaper and faster (arXiv preprint)
An arXiv preprint compares Jev, a decision language model that outputs structured judgments, against frontier and open-weight LLM judges across four clinical-evaluation use cases. On 118 pediatric answers Jev's agreement with physician ordinal ratings (Gwet's AC2 0.50; expected 0.53) matched or exceeded seven of 12 LLM judges (including Claude Opus 5.5 at 0.49 and GPT-6 at 0.38); Jev matched or exceeded several judges on HealthBench and MedHELM tasks and matched GPT-5.4 on 91% of HealthBench Professional criteria, while frontier judges cost 73–294× more and took 7–57× longer.