Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #32503

Can We Trust AI Evaluation? (NeurIPS 2026 workshop)

Related event timeline, sources and context from the news index.

EVENT TIMELINE

1

RESEARCH · 1 SOURCE · Apple Machine Learning Research

Evaluating the Evaluators: LLM judges often misalign with humans on personalized writing style (NeurIPS 2026 workshop)

This paper (accepted to the Can We Trust AI Evaluation? workshop at NeurIPS 2026) studies how well LLMs function as judges for personalized long-form writing tasks. The authors generated text with four LLMs across three tasks, collected human pairwise preferences, and compared those to LLM judge pairwise rankings; they report weak correlation between judge quality and general ability (MMLU ρ ≤ 0.2), performance that varies by task and evaluation objective, better consistency from paired comparisons than independent scoring, and that the highest-performing LLMs are not the most human-aligned judges.

6.0