# NVIDIA H100 The H100 is the accelerator on which most of the models that defined the current era were trained. Its generation introduced the features that are now assumed in every serious accelerator: a transformer-oriented arithmetic engine, eight-bit floating point support, and a fabric generation capable of binding eight devices into one memory domain at usable bandwidth. > [!note] The one-sentence version > The part that made training frontier transformers economical, and therefore the reference against which everything since is measured. ## What the generation added **Eight-bit floating point arithmetic in hardware.** Lower precision multiplies both effective memory bandwidth and arithmetic throughput, and hardware support is what moved it from a research technique to a production default; see [[wiki/Quantization|Quantization]]. **A transformer-oriented execution engine** that manages precision dynamically across a model's layers, keeping sensitive operations at higher precision while the bulk of the arithmetic runs lower. **Stacked memory at substantially higher bandwidth** than the preceding generation, which matters because serving and much of training are bandwidth-bound; see [[wiki/High-Bandwidth Memory|High-Bandwidth Memory]] and [[wiki/Memory Wall|Memory Wall]]. **A fabric generation** raising per-accelerator aggregate bandwidth enough that an eight-device domain behaved as one accelerator for tensor-parallel work; see [[wiki/NVLink|NVLink]]. **Partitioning,** allowing one physical accelerator to be divided into isolated instances, which is how a single device serves several smaller tenants predictably. ## Its place now As of October 2026 this generation is no longer the frontier, and it remains in very large-scale production service. Its successors raised memory capacity and bandwidth and then changed the integration level entirely with rack-scale systems; see [[wiki/NVIDIA H200|NVIDIA H200]], [[wiki/NVIDIA B200|NVIDIA B200]], and [[wiki/Rack-Scale Computing|Rack-Scale Computing]]. The cascade described in [[wiki/Depreciation of Compute Assets|Depreciation of Compute Assets]] is visible here in real time: parts retired from frontier training carry years of productive service in inference, fine-tuning, research, and academic work, which is the mechanism by which capable compute becomes broadly affordable a few years after it is new. That cascade is the reason this entry matters to a reader thinking about access rather than about frontier capability. A university department or a mid-sized company obtaining this generation today is obtaining hardware that trained the models everyone is now using. ## Relationships - **mounted on:** [[wiki/NVIDIA HGX|NVIDIA HGX]] - **succeeded by:** [[wiki/NVIDIA H200|NVIDIA H200]] · [[wiki/NVIDIA B200|NVIDIA B200]] · [[wiki/NVIDIA B300|NVIDIA B300]] - **connected by:** [[wiki/NVLink|NVLink]] · [[wiki/InfiniBand|InfiniBand]] - **depends on:** [[wiki/High-Bandwidth Memory|High-Bandwidth Memory]] · [[wiki/Advanced Packaging|Advanced Packaging]] · [[wiki/TSMC|TSMC]] - **programmed through:** [[wiki/CUDA|CUDA]] - **cascades per:** [[wiki/Depreciation of Compute Assets|Depreciation of Compute Assets]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** hardware eight-bit floating point support, a transformer-oriented engine managing precision dynamically, higher stacked memory bandwidth than the preceding generation, a fabric generation supporting an effective eight-device memory domain, and device partitioning into isolated instances. - **Established (datestamped):** as of October 2026 this generation is no longer the frontier part and remains in very large-scale production service. - **Strongly indicated:** the cascade of retired frontier parts into inference, research, and academic use as the principal mechanism broadening compute access; the pattern is well documented in semiconductor equipment and is still accruing for accelerators. ## Sources - Public accelerator architecture and specification documentation, as of October 2026.