Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

NEWS · COMPANIES · #3880

Airbnb uses cost-per-task and CloudWatch metrics to run AirChat coding agents on Amazon Bedrock

Airbnb routes its internal coding agents (AirChat and AirChat Remote) through an LLM gateway into Amazon Bedrock and evaluates models using task-specific evals that report score and list-price cost per task. Production monitoring in Amazon CloudWatch (latency, token counts, throttles, EstimatedTPMQuotaUsage) is compared with eval results to set model defaults, effort ceilings, and to diagnose latency, throttling, and cost changes.

KEY POINTS

  1. Airbnb routes its internal coding agents (AirChat and AirChat Remote) through an LLM gateway into Amazon Bedrock and evaluates models using task-specific evals that report score and list-price cost per task.
  2. Production monitoring in Amazon CloudWatch (latency, token counts, throttles, EstimatedTPMQuotaUsage) is compared with eval results to set model defaults, effort ceilings, and to diagnose latency, throttling, and cost changes.
  3. This shows a practical, production-ready approach to selecting and operating LLMs by combining per-task cost-aware evals with CloudWatch observability on Amazon Bedrock, which many teams can adopt to control latency, capacity, and spend.

WHY IT MATTERS

This shows a practical, production-ready approach to selecting and operating LLMs by combining per-task cost-aware evals with CloudWatch observability on Amazon Bedrock, which many teams can adopt to control latency, capacity, and spend.

SOURCES & TIMELINE

1