Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

NEWS · COMPANIES · #3670

Amazon Bedrock adds explicit prompt caching for open-weight models

Amazon Web Services announced that open-weight models on Amazon Bedrock now support explicit prompt caching (in addition to implicit caching). Prompt caching lets Bedrock reuse computed state for matching prompt prefixes to reduce input processing, lower input billing rates for cache reads, and cut response latency; developers use the Chat Completions API and should consult each model’s card for supported caching modes, minimum prefix lengths, breakpoints, and TTLs.

KEY POINTS

  1. Amazon Web Services announced that open-weight models on Amazon Bedrock now support explicit prompt caching (in addition to implicit caching).
  2. Prompt caching lets Bedrock reuse computed state for matching prompt prefixes to reduce input processing, lower input billing rates for cache reads, and cut response latency; developers use the Chat Completions API and should consult each model’s card for supported caching modes, minimum prefix lengths, breakpoints, and TTLs.
  3. This reduces compute, latency, and input costs for applications that repeatedly send the same context and gives developers explicit control to avoid unnecessary cache writes (which can affect billing).

WHY IT MATTERS

This reduces compute, latency, and input costs for applications that repeatedly send the same context and gives developers explicit control to avoid unnecessary cache writes (which can affect billing).

SOURCES & TIMELINE

1