Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

RESEARCH · RESEARCH · #3630

Study (arXiv:2611.17639v1) examines what LLM agents act on vs. what users say

The paper uses presupposition theory to analyze when LLM agents proceed with actions (e.g., cancel reservations, delete folders) based on tool outputs versus user statements. It shows that agents frequently act without asking when lookups cannot distinguish one vs. multiple records, that listings stop most agents whereas simple counts do not, that user claims (e.g., “it is empty”) can reduce acting, and that latest Claude and GPT models tested opened a counted second record then never acted; a retail benchmark with same-name same-zip customers produced wrong-account writes in about a quarter of conversations unless lookups checked competing records or street data.

KEY POINTS

  1. The paper uses presupposition theory to analyze when LLM agents proceed with actions (e.g., cancel reservations, delete folders) based on tool outputs versus user statements.
  2. It shows that agents frequently act without asking when lookups cannot distinguish one vs.
  3. multiple records, that listings stop most agents whereas simple counts do not, that user claims (e.g., “it is empty”) can reduce acting, and that latest Claude and GPT models tested opened a counted second record then never acted; a retail benchmark with same-name same-zip customers produced wrong-account writes in about a quarter of conversations unless lookups checked competing records or street data.

WHY IT MATTERS

Shows that tool presentation (list vs count vs check) and user claims materially change agent safety and correctness, implying lookups and evaluations must expose competing records to avoid harmful mistaken actions.

SOURCES & TIMELINE

1