Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #33775

EMNLP 2026

Related event timeline, sources and context from the news index.

EVENT TIMELINE

1

RESEARCH · 1 SOURCE · Hugging Face

XYBench benchmark released to test models' handling of the 'XY problem' (EMNLP 2026 Best Resource Paper)

XYBench is an 8,115-question benchmark (from Stack Overflow, Stack Exchange, curated examples, and WikiHow-generated prompts) that tests whether assistants recognize when users ask about a mistaken approach (Y) instead of their actual goal (X). Developed with Carnegie Mellon, NVIDIA, and Johns Hopkins and judged automatically by Sonnet 4.5, XYBench evaluated models including Claude Sonnet 4.6, GPT-5, Gemini 3.1 Pro, Qwen3.5-9B, and DeepSeek-V4 Pro, finding they often answer the asked approach rather than redirecting to solutions for the underlying goal; the paper won the Best Resource Paper award at EMNLP 2026.

7.0