Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

TOPIC · ENTITY #28949

tool-using judge

Related event timeline, sources and context from the news index.

EVENT TIMELINE

1

RESEARCH · 1 SOURCE · arXiv cs.AI

Grounded but Not Immune: confirmation bias in tool-using Agent-as-a-Judge (arXiv:2611.05662v1)

The paper (arXiv:2611.05662v1) shows that Agent-as-a-Judge evaluators that use tools and observe full agent trajectories remain vulnerable to confirmation bias: a single false pre-screening note attributed to an authority flips up to 40% of correct verdicts across four judges and five datasets, even when judges gather decisive evidence. The authors analyze three failure modes—over-trust, hallucination, and under-investigation—find Claude-based judges mostly fail by over-trust, and report that lightweight mitigations reduce but do not eliminate the bias.

7.0