Rapidly scaling online storage to serve over 1 billion ChatGPT users

Rare numbers from inside ChatGPT’s storage layer. OpenAI’s “Habitat” handles 70M+ requests per second over 500+ PB in 40 regions, sitting in front of Cosmos DB, Valkey and blob storage. My favorite bits: FIFO connection pooling to avoid metastable failures, and a Rust rewrite by two engineers that got 6x better CPU and 15x better memory efficiency than the Python version.