Shared vs. Dedicated Inference: embedding and reranking costs depend on request shape, not RPM alone
The note argues that request shape (request size, token volume, and rerank candidate count) and traffic patterns matter more than raw requests-per-minute for sizing and economics of embedding and reranking workloads. Dedicated provisioned inference is generally more cost-effective for steady, predictable loads, while shared consumption-based inference fits variable or bursty traffic; the break-even point depends on workload details including latency and usable throughput, not just headline prices.