NEWS · COMPANIES · #1040
Perplexity replaces DynamoDB with in-house CobbleDB to cut batch-read latency 5x
Perplexity moved its core search serving tier from Amazon DynamoDB to CobbleDB, an internally developed Rust-based distributed key-value store, splitting durable storage (Pillar) from hot serving (CobbleDB) and using Lorry to aggregate batch files in S3. The change reportedly reduced median batch-read latency from 31.4 ms to 5.60 ms, cut p99 tail latency from 123 ms to 24.2 ms, sustained synthetic throughput up to 500k requests/sec, and lowered overall storage costs by at least 20%.
KEY POINTS
- Perplexity moved its core search serving tier from Amazon DynamoDB to CobbleDB, an internally developed Rust-based distributed key-value store, splitting durable storage (Pillar) from hot serving (CobbleDB) and using Lorry to aggregate batch files in S3.
- The change reportedly reduced median batch-read latency from 31.4 ms to 5.60 ms, cut p99 tail latency from 123 ms to 24.2 ms, sustained synthetic throughput up to 500k requests/sec, and lowered overall storage costs by at least 20%.
- This shows how AI-centric retrieval workloads can drive firms to build specialized, eventually-consistent serving systems to cut latency and cloud cost at production scale.
WHY IT MATTERS
This shows how AI-centric retrieval workloads can drive firms to build specialized, eventually-consistent serving systems to cut latency and cloud cost at production scale.