OpenAI describes changes to Habitat’s Azure Cosmos DB storage, including selective replication and caching, that lowered median global read latency twelvefold.
OpenAI says it has reduced the median global read latency of Habitat, its online storage platform, from roughly 120 milliseconds to roughly 10 milliseconds. The company attributes the twelvefold improvement to a combination of placing selected data closer to users, adding distributed caching and rewriting part of the service in Rust.
The engineering account, published October 23, 2026, describes a platform supporting OpenAI products across 40 geographic regions and storing more than 500 petabytes. Habitat serves products including ChatGPT, the API and Codex, and relies on Azure Cosmos DB for online storage. OpenAI says that as its services expanded globally, compute and stored data were often in different regions, making cross-region requests a source of delay.
The company’s approach is selective rather than replicating everything everywhere. It says particularly latency-sensitive information, such as authentication metadata, is replicated across multiple Azure regions; some accounts span all 40 regions where OpenAI serves traffic, while others cover at least one region per continent. Capacity constraints limit broad replication to a small set of datasets. For other frequently accessed information, Habitat uses a distributed look-aside cache in Valkey servers located near request-processing infrastructure.
OpenAI also details changes to how Habitat shares database capacity among workloads. After trying dedicated containers for individual teams, the company returned to multi-tenancy by default, working with Azure on measures including throttling unusually hot keys and assigning lower priority to offline work such as bulk inserts and data extracts. The goal is to reduce interference between workloads while avoiding the capacity-planning burden of isolated containers.
The latency figure reflects the combined changes, not a single optimization. OpenAI says the slowest requests still involve uncached data fetched from underlying storage, sometimes across regions. It plans to address data placement further as more capacity comes online and storage efficiency improves.