RELEASE · COMPANIES · #2573
Cloudflare launches Auto Router public beta in AI Gateway to rate requests by stakes and other metrics
Cloudflare published Auto Router in public beta via AI Gateway: a two-stage router that builds a candidate model pool then runs a multi-head classifier on Workers AI to assign probabilities across 14 task categories and score requests 1–5 on complexity, ambiguity, stakes and context dependence, using only the newest turns (up to 8 KiB). The router combines those scores with benchmark results (quoted formula: "utility = expected quality - adaptive cost penalty"), exposes routing headers for auditability, supports per-request allowed-models headers, is free in beta, and currently lacks WebSocket support.
KEY POINTS
- Cloudflare published Auto Router in public beta via AI Gateway: a two-stage router that builds a candidate model pool then runs a multi-head classifier on Workers AI to assign probabilities across 14 task categories and score requests 1–5 on complexity, ambiguity, stakes and context dependence, using only the newest turns (up to 8 KiB).
- The router combines those scores with benchmark results (quoted formula: "utility = expected quality - adaptive cost penalty"), exposes routing headers for auditability, supports per-request allowed-models headers, is free in beta, and currently lacks WebSocket support.
- This changes how teams route requests across models at the edge by scoring requests for risk and expected gain, making routing decisions auditable but raising concerns about using only recent turns and pool composition.
WHY IT MATTERS
This changes how teams route requests across models at the edge by scoring requests for risk and expected gain, making routing decisions auditable but raising concerns about using only recent turns and pool composition.