Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

TOPIC · ENTITY #31518

autorater

Related event timeline, sources and context from the news index.

EVENT TIMELINE

1

RESEARCH · 1 SOURCE · Google

Google and Artificial Analysis launch the Multimodal Knowledge Agent (MKA) benchmark

MKA is a new benchmark, developed with Artificial Analysis, designed to evaluate how agents reason across visually complex files in realistic knowledge-work tasks; it includes a 200-task private evaluation set (plus 10 public tasks on Hugging Face), an open-source agent harness called Stirrup, and an autorater for automated scoring. The report shows large efficiency and perception differences across models, and notes Gemini 4 Carbon ranks #2 on quality while completing tasks at roughly 6× lower cost than the top-scoring model.

7.0