prompt caching

2026-10-26

CODING · 1 SOURCE · AWS Machine Learning

Amazon Bedrock documents prompt-caching patterns for running OpenAI models in agents

An Amazon Bedrock post explains implicit and explicit prompt caching for agent workloads running OpenAI models, showing how to save the model’s prefill (KV) state for reusable prefixes (e.g., tools and instructions) using prompt_cache_breakpoint and prompt_cache_key to cut token costs and latency; it notes caching requires exact identical text and a reusable prefix of at least 1,024 tokens. The post also discusses combining implicit caching (Bedrock-chosen breakpoints) with explicit markers to share starting state across sessions and subagents.

5.0

2026-09-22

MODELS · 1 SOURCE · OpenAI

GPT-6 improves prompt caching

GPT-6 introduces improved prompt caching with higher cache hit rates, new diagnostics, explicit breakpoints, and user controls intended to reduce latency and costs. These features target prompt-heavy workflows that can benefit from more efficient reuse of prompt context.

6.0

2026-09-22

MODELS · 4 SOURCES · AWS Machine Learning · TechCrunch AI · OpenAI · The Decoder

OpenAI's GPT-6 Sol and GPT-6 Luna now generally available on Amazon Bedrock

OpenAI's GPT-6 Sol and GPT-6 Luna are now generally available on Amazon Bedrock, running on Bedrock's high-performance inference engine and offered at lower API pricing than their GPT-5.6 predecessors. GPT-6 Sol is positioned for demanding, recurring development and operations tasks with improved coding, reasoning and factuality (OpenAI reports ~50% fewer factual mistakes vs GPT-5.6 Sol), while GPT-6 Luna is optimized for high-volume, repeatable workloads like extraction, classification and summarization; Bedrock supports explicit prompt caching plus IAM-based access control and invocation auditing.

8.0

2026-09-15

CODING · 1 SOURCE · AWS Machine Learning

Optimizing cost and latency with Amazon Bedrock prompt caching

An AWS Machine Learning post describes prompt caching in Amazon Bedrock, saying it can cut input token costs by up to 90% when the same context is repeatedly sent to foundation models. The article walks through six practical prompt caching scenarios for the Converse API: message content, system prompt, tool definition, mixed TTL, tenant isolation, and LangChain integration.

4.0