Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

FUNDING · COMPANIES · #3727

Datadog adds Online Experiments to Agent Observability to test agents on production traffic

Datadog has extended its Agent Observability platform with an Online Experiments capability that lets teams A/B agent versions on real production traffic and compare evaluation scores alongside operational metrics (latency, token consumption, cost, errors). InfoQ reports Datadog used the feature to split traffic between GPT-5.6 Luna and GPT-6 Luna, finding a reported 44.3% estimated cost reduction for the newer model but also signals of increased execution time and more invalid tool parameters in traces.

KEY POINTS

  1. Datadog has extended its Agent Observability platform with an Online Experiments capability that lets teams A/B agent versions on real production traffic and compare evaluation scores alongside operational metrics (latency, token consumption, cost, errors).
  2. InfoQ reports Datadog used the feature to split traffic between GPT-5.6 Luna and GPT-6 Luna, finding a reported 44.3% estimated cost reduction for the newer model but also signals of increased execution time and more invalid tool parameters in traces.
  3. Bringing controlled A/B experiments, production telemetry, and traces together helps teams detect regressions and trade-offs that offline tests can miss when deploying AI agents in real conditions.

WHY IT MATTERS

Bringing controlled A/B experiments, production telemetry, and traces together helps teams detect regressions and trade-offs that offline tests can miss when deploying AI agents in real conditions.

SOURCES & TIMELINE

1