# S3-Compatible Object Storage Object storage holds whole items retrieved by name, over HTTP, in a flat namespace, with no file system semantics at all: no directories, no partial writes, no file handles. Giving up those conveniences buys essentially unlimited scale, strong durability, and a very low cost per byte. The interface for doing so, originally one provider's, became a de facto industry standard implemented by dozens of systems, which is a story worth understanding on its own. > [!note] The one-sentence version > Put a blob, get a blob, by name, over HTTP; abandon file system semantics and gain unlimited scale and durability. ## Why it exists File systems promise a great deal: hierarchical names, in-place modification, locking, concurrent partial access, consistent metadata. Those promises are expensive to keep across thousands of machines in several buildings. Object storage declines most of them. An object is written whole, read whole or by byte range, identified by a key in a flat bucket, and replaced rather than modified. What remains is simple enough to distribute almost arbitrarily, which is how a single bucket holds exabytes and survives the loss of entire facilities. ## Prerequisites - **Bucket and key:** the container and the object's name within it; the apparent folders in a key are a display convention over a flat namespace. - **Immutability:** objects are replaced rather than edited, which simplifies distribution and makes versioning natural. - **Eventual and strong consistency:** whether a read immediately after a write is guaranteed to see it; the major implementations now provide strong read-after-write consistency, which removed a long-standing source of application complexity. - **Durability through erasure coding:** fragments with parity spread across devices, racks, and sites, so that loss of many components loses no data. - **Storage classes and lifecycle policy:** tiers trading retrieval latency and cost, with automatic transitions by age. - **Presigned URLs and bucket policy:** time-limited delegated access and the access-control model. - **Multipart upload:** writing a very large object as parallel parts, which is how throughput is achieved against a single-object-at-a-time interface. ## Why the interface became the standard One provider's interface was adopted by competitors, open-source projects, on-premises appliances, and specialist systems, until supporting it became a requirement for any storage product. The reason is worth noting: the interface is simple, it is adequate, and it was first with sufficient scale to become what applications were written against. Every tool, library, and framework in the data ecosystem speaks it. For buyers the consequence is excellent, because workloads and data become portable across implementations and across providers, which is a rare instance of genuine interoperability arriving by adoption rather than by committee. ## What it holds in an AI facility The durable corpus: raw training datasets before preprocessing, archived model checkpoints and released weights, logs and telemetry, retrieval corpora, and anything that must survive. The working set is then staged onto fast parallel storage or local flash for the duration of a run; see [[wiki/Parallel File System|Parallel File System]] and [[wiki/Storage Tiering|Storage Tiering]]. The division is clean and worth remembering: object storage is where data lives, and parallel file systems are where data is worked on. ## Relationships - **tiered with:** [[wiki/Parallel File System|Parallel File System]] · [[wiki/NVMe SSD|NVMe SSD]] · [[wiki/Storage Tiering|Storage Tiering]] - **holds:** [[wiki/Checkpointing|Checkpointing]] archives · [[wiki/Retrieval-Augmented Generation|Retrieval-Augmented Generation]] corpora - **accessed by:** [[wiki/PyTorch|PyTorch]] · [[wiki/OCI Container|OCI Container]] registries - **part of:** [[wiki/Data Fabric and Storage|Data Fabric and Storage]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** flat bucket-and-key namespace with whole-object write semantics; durability through erasure coding across devices, racks, and sites; storage classes and lifecycle policies; presigned URLs and bucket policy access control; multipart upload for parallel throughput; strong read-after-write consistency in major current implementations. - **Established:** the interface's adoption across competing providers, open-source projects, and on-premises systems as a de facto standard, and the resulting portability of tooling and data. - **Established:** the production pattern of durable object storage with working sets staged onto fast parallel storage or local flash. ## Sources - Object storage interface documentation and distributed storage durability practice, as current October 2026.