# Storage Tiering Data in an AI facility lives at several levels at once, each larger, slower, and cheaper per byte than the one above it. Tiering is the practice of deciding what sits where and of moving data between levels so that the expensive processors are never the component waiting. The hierarchy is the same one computer architecture has always had, extended outward until it spans buildings. > [!note] The one-sentence version > Keep what is being used closest and fastest, keep everything else where it is cheap, and move data between them before anyone needs it. ## The levels 1. **Accelerator memory.** Stacked memory on the accelerator package; terabytes per rack, the fastest and scarcest storage in the building, holding model weights and activations; see [[wiki/High-Bandwidth Memory|High-Bandwidth Memory]]. 2. **Host memory.** Two to three terabytes per server, used for staging batches, overflow, and buffering; see [[wiki/Host CPU|Host CPU]]. 3. **Local flash.** A few to tens of terabytes per server, holding the operating environment, a working subset, cached weights, and checkpoint staging; see [[wiki/NVMe SSD|NVMe SSD]]. 4. **Cluster flash.** A parallel file system of petabytes, serving every node at aggregate bandwidth; the working tier for active datasets and checkpoints; see [[wiki/Parallel File System|Parallel File System]]. 5. **Object storage.** Exabyte-capable, durable, inexpensive; the durable corpus and archive; see [[wiki/S3-Compatible Object Storage|S3-Compatible Object Storage]]. 6. **Cold archive.** Still cheaper classes with retrieval delays measured in minutes to hours, for material that must be kept and will rarely be read. Each step down multiplies capacity and divides cost per byte, while latency and bandwidth move the other way. The total ratio from top to bottom spans many orders of magnitude in every dimension, which is why no single tier can serve the whole need. ## How data moves **Staging before a run.** The dataset for the next job is copied from object storage to the parallel file system, and sometimes onto local flash, while the previous job is still running. Prefetching in advance is what separates a well-run cluster from one whose accelerators wait at every job boundary. **Caching during a run.** Frequently read items are retained in local flash or host memory; a transparent caching layer in front of object storage is a common arrangement, giving object-storage economics with closer-to-flash latency for repeat reads. **Checkpoint flush.** Checkpoints are written to the fastest tier that can absorb the burst, then moved asynchronously to durable storage, so that the training run resumes without waiting for the archive; see [[wiki/Checkpointing|Checkpointing]]. **Lifecycle transitions.** Policies move aging data down automatically, which is how a facility accumulates history without accumulating cost. ## The governing principle Every tier exists because the tier above it is too expensive to hold everything and the tier below is too slow to serve the work. The engineering aim is that the processors see only the fast tiers, and all movement happens in advance of need. When that is achieved, [[wiki/Utilization|utilization]] stays high and the capital in the building earns; when it is not, accelerators idle while data moves, and the loss is invisible on every dashboard except the one that matters. ## Relationships - **spans:** [[wiki/High-Bandwidth Memory|High-Bandwidth Memory]] · [[wiki/NVMe SSD|NVMe SSD]] · [[wiki/Parallel File System|Parallel File System]] · [[wiki/S3-Compatible Object Storage|S3-Compatible Object Storage]] - **serves:** [[wiki/Utilization|Utilization]] · [[wiki/Checkpointing|Checkpointing]] - **moved by:** [[wiki/RDMA|RDMA]] · [[wiki/NVMe over Fabrics|NVMe over Fabrics]] - **part of:** [[wiki/Data Fabric and Storage|Data Fabric and Storage]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** the tier structure from accelerator memory through host memory, local flash, cluster flash, object storage, and cold archive, with capacity and cost per byte rising and latency and bandwidth falling at each step; prefetching and staging ahead of job start; transparent caching in front of object storage; asynchronous checkpoint flush to durable tiers; lifecycle policy transitions. - **Established:** two to three terabytes of host memory and multi-terabyte local flash in published accelerator server building blocks. - **Strongly indicated:** data movement at job boundaries as a significant and frequently unmeasured source of accelerator idleness; operator practice treats it as primary and published quantification is limited. ## Sources - High-performance storage architecture practice for AI training clusters, as current October 2026. - Public accelerator server specifications, as of October 2026.