Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

TOPIC · ENTITY #31980

simulated operator

Related event timeline, sources and context from the news index.

EVENT TIMELINE

1

RESEARCH · 1 SOURCE · arXiv cs.AI

Ask2Decide benchmark tests whether LLM agents request decision-relevant information

Ask2Decide is a new benchmark (arXiv:2611.13015v1) of 43 planning tasks that evaluates whether LLM agents ask for missing facts that change optimal plans. The authors collected 79 interaction sessions from 19 human participants, calibrated a simulated operator to match human reply patterns, and found that four frontier LLMs produced infeasible initial plans on 58–98% of tasks when information was missing and uncovered decision-changing facts only marginally more often than irrelevant facts (39–52% vs 32–41%).

6.0