# Accelerator An accelerator is a processor built to do one kind of arithmetic extremely well. In this decade that arithmetic is matrix multiplication: taking two grids of numbers and producing a third, billions of times a second. Neural networks are, at the level of the machine, almost entirely that operation; a processor that performs it in bulk performs intelligence work in bulk. Everything else in the facility, the power, the cooling, the fabric, the floor, exists to keep accelerators running. > [!note] The one-sentence version > A general processor does a few things quickly in sequence; an accelerator does one thing, in parallel, tens of thousands of times at once. ## Why it exists A central processing unit is built for choice. It has a handful of powerful cores optimized to run arbitrary instructions in sequence, predict branches, and respond quickly to the unexpected; that design is why a computer can run an operating system, a browser, and a database at the same time. Neural network arithmetic wants the opposite: the same simple operation applied across enormous arrays, with the order of operations known in advance. Thousands of small arithmetic units, fed from wide memory, deliver vastly more of that work per watt and per square millimeter than a few large flexible ones. The deep lesson of the last fifteen years is that when a workload becomes economically important enough, the hardware reorganizes around its shape. ## Prerequisites - **Tensor:** an array of numbers with one or more dimensions; the data structure neural networks are written in. - **Matrix multiplication:** the operation at the center of nearly every layer of a modern model. - **Training and inference:** training adjusts a model's parameters from data; inference uses the finished parameters to answer a request. Both are matrix work, with different ratios of arithmetic to memory traffic; see [[wiki/Memory Wall|Memory Wall]]. - **Precision:** how many bits represent each number. Lower precision costs less memory and bandwidth per value; see [[wiki/Quantization|Quantization]]. - **Host:** the ordinary processor that boots the machine, runs the operating system, and feeds the accelerators; see [[wiki/Host CPU|Host CPU]]. ## The families **Graphics processing units.** Built originally to shade pixels in parallel, these proved to be general parallel arithmetic engines, and a 2012 image-recognition result trained on them is reasonably called the field's opening event. They carry thousands of small cores plus dedicated matrix units, pair with stacked memory, and remain programmable; a model architecture invented next year will run on hardware bought this year. That flexibility is the central argument for them. **Custom application-specific integrated circuits.** A function committed to silicon, hard-wired and unchangeable once etched. The reward is efficiency: fewer transistors spent on generality, more on the work. The cost is flexibility and a design effort running from tens to hundreds of millions of dollars before the first part exists. Large operators who know their own workloads build them; the first widely recognized custom AI chip of this kind was coined in 2015, and the family has since grown to serve several hyperscale operators, with specialist design houses supplying much of the physical implementation and interconnect expertise. **Neural processing units at the edge.** The same idea at low power, integrated into a phone, laptop, car, camera, or robot alongside the main processor. Work performed on the device stays on the device, uses less silicon, and costs less than a round trip to a data center; see [[wiki/Edge Inference|Edge Inference]]. **Reconfigurable logic.** Field-programmable gate arrays can be rewired after manufacture, offering hardware-level specialization without committing to a mask set. They trade peak performance and efficiency against that flexibility, and they are the rational choice at volumes too low to amortize a custom design. **Wafer-scale and inference-specialist architectures.** Several firms pursue designs outside these categories, including processors occupying most of a wafer and architectures built specifically for low-latency token generation. See [[wiki/Inference Accelerator|Inference Accelerator]]. ## Stack position - **Upstream:** [[wiki/Foundry|foundry]] capacity, [[wiki/Advanced Packaging|advanced packaging]], and [[wiki/High-Bandwidth Memory|memory]] supply, which together set how many exist. - **Downstream:** [[wiki/Compute Tray|trays]], [[wiki/Rack-Scale Computing|racks]], fabrics, and every service above them. - **Function and vendor:** the function is central and the suppliers are several; one vendor holds the large majority of data-center share, with a credible second GPU supplier and a growing custom-silicon field. - **Degraded case:** the same work requires far more machines, far more power, and far more floor, at which point the economics that justify the buildout weaken. ## Lineage and trajectory Graphics hardware became scientific hardware, then became the substrate of machine learning; a transformer architecture published in 2017 then concentrated the entire field onto one set of operations, which is exactly the condition under which specialized silicon wins. Accelerators now ship as rack-scale systems rather than cards, because a single model exceeds one device's memory. The trajectory runs toward more memory per package, lower numeric precision, tighter coupling between accelerators inside a rack, and a widening share of total output going to inference rather than training as deployed use grows. ## Relationships - **depends on:** [[wiki/Chiplet|Chiplet]] · [[wiki/High-Bandwidth Memory|High-Bandwidth Memory]] · [[wiki/Advanced Packaging|Advanced Packaging]] · [[wiki/Host CPU|Host CPU]] - **integrated into:** [[wiki/Compute Tray|Compute Tray]] · [[wiki/Rack-Scale Computing|Rack-Scale Computing]] - **connected by:** [[wiki/NVLink|NVLink]] · [[wiki/UALink|UALink]] · [[wiki/Infinity Fabric|Infinity Fabric]] - **constrained by:** [[wiki/Memory Wall|Memory Wall]] · [[wiki/Supply Chain Chokepoint|Supply Chain Chokepoint]] - **variants:** [[wiki/Inference Accelerator|Inference Accelerator]] · [[wiki/Edge Inference|Edge Inference]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** the architectural contrast between few sequential cores and many parallel matrix units; the 2012 image-recognition result as the field's opening demonstration; the 2017 transformer architecture as the convergence point; custom design costs from tens to hundreds of millions of dollars; the first custom AI chip of this class dating to 2015; AI chips passing a quarter of all chips sold during 2025; reconfigurable logic's flexibility-against-efficiency trade and the volume threshold at which custom silicon becomes cheaper. - **Established (datestamped):** as of October 2026, one supplier holds well above ninety percent of data-center accelerator share. - **Plausible:** AI chips exceeding half of all chips sold toward the end of the decade; this is an industry expectation rather than a measured outcome. - **Unresolved:** the eventual share of custom silicon against merchant GPUs; analyst projections diverge, and the deciding evidence is multi-year shipment data. ## Sources - [How Nvidia GPUs Compare To Google's And Amazon's AI Chips](https://www.cnbc.com/) — CNBC, October 2026; processor architecture contrast, custom design economics, edge accelerators, reconfigurable logic trade-offs, chip-share figures. - [How AI Is Pushing the Semiconductor Supply Chain to the Limit](https://www.bloomberg.com/) — Bloomberg Originals, October 2026; AI share of chips sold and its projected trajectory.