# NVIDIA Grace Blackwell Grace Blackwell pairs a custom server processor with accelerators on one module, joined by a coherent interconnect rather than by the general expansion bus. The arrangement treats the host and the accelerator as one computer with one memory space, and it is the clearest expression of a trend visible throughout this collection: every boundary in the machine is being tightened, and the host boundary was next. > [!note] The one-sentence version > Put the processor and the accelerators on the same module, coherently, and the host stops being a bottleneck and becomes part of the accelerator. ## Why couple them An accelerator's host does real work: it runs the training process, prepares and feeds data, coordinates collectives, and holds model state too large for accelerator memory; see [[wiki/Host CPU|Host CPU]]. Traditionally that host sits across the expansion bus, which is adequate for control and modest for data; see [[wiki/PCIe|PCIe]]. Replacing that link with a coherent interconnect at an order of magnitude more bandwidth changes what is architecturally possible: the accelerator can reach host memory as an extension of its own, so a model's optimizer state or a very large embedding table can live in the host's larger, cheaper memory while the accelerator works from it. The memory hierarchy gains a usable tier; see [[wiki/Storage Tiering|Storage Tiering]]. Designing the processor rather than buying one also lets the memory system, the interconnect, and the power envelope be specified for this single purpose, which is the same vertical-integration logic that produces custom accelerators at the largest operators; see [[wiki/Hyperscaler|Hyperscaler]]. ## Where it sits Modules of this kind are the building block of the vendor's rack-scale systems, in which seventy-two accelerators and their hosts share one fabric, one power domain, and one cooling loop; see [[wiki/NVIDIA NVL72|NVIDIA NVL72]] and [[wiki/Rack-Scale Computing|Rack-Scale Computing]]. The competing architecture reaches the same coupling with a general-purpose server processor line and a consortium fabric, which is a genuine strategic difference: one path integrates vertically, the other standardizes horizontally; see [[wiki/AMD Helios|AMD Helios]] and [[wiki/UALink|UALink]]. ## Relationships - **combines:** [[wiki/Host CPU|Host CPU]] · [[wiki/NVIDIA B200|NVIDIA B200]] - **replaces for host attachment:** [[wiki/PCIe|PCIe]] - **installed in:** [[wiki/NVIDIA NVL72|NVIDIA NVL72]] · [[wiki/Compute Tray|Compute Tray]] - **succeeded by:** [[wiki/NVIDIA Vera Rubin|NVIDIA Vera Rubin]] - **compared with:** [[wiki/AMD Helios|AMD Helios]] · [[wiki/Infinity Fabric|Infinity Fabric]] - **part of:** [[wiki/Compute and Accelerator Systems|Compute and Accelerator Systems]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** a custom server processor paired with accelerators over a coherent interconnect at substantially higher bandwidth than the general expansion bus, presenting a shared memory space; the host processor's role as data-pipeline and collective coordinator; modules of this kind as the building block of the vendor's rack-scale systems. - **Established:** the competing rack-scale architecture pairing a general-purpose server processor line with a consortium scale-up fabric. - **Strongly indicated:** coherent host attachment adding a usable memory tier for optimizer state and large tables; the capability is documented and workload-level benefit varies. ## Sources - [How Nvidia GPUs Compare To Google's And Amazon's AI Chips](https://www.cnbc.com/) — CNBC, October 2026; the custom host processor within the rack-scale module. - Public accelerator platform documentation, as of October 2026.