# Software and Orchestration Between the hardware and the people using it sits a stack of software that decides, in practice, which hardware is usable at all. Silicon specifications set a ceiling; this layer determines how much of it anyone reaches. It is also where the industry's competitive position is currently decided, which makes it worth as much attention as the chips. > [!note] The one-sentence version > Drivers, compilers, kernels, frameworks, containers and schedulers; the silicon is a ceiling and this is how close anyone gets to it. ## The rungs 1. **Driver and runtime.** The kernel driver and user-space runtime that present the accelerator to software, with the version coupling that makes fleet upgrades a coordinated operation; see [[wiki/OCI Container|OCI Container]]. 2. **Compiler and kernel libraries.** The compiler for accelerator code and the libraries of hand-tuned kernels for linear algebra, transforms, attention, and collectives; this accumulation, rather than any single capability, is the incumbent's position; see [[wiki/CUDA|CUDA]] · [[wiki/ROCm|ROCm]]. 3. **Collective communication.** The libraries implementing group operations over whatever fabric is present, topology-aware; see [[wiki/Collective Communication|Collective Communication]]. 4. **Frameworks.** Where models are actually written, and the layer at which hardware becomes substitutable; see [[wiki/PyTorch|PyTorch]]. 5. **Packaging.** Immutable images carrying frameworks, libraries, and dependencies; see [[wiki/OCI Container|OCI Container]]. 6. **Orchestration and scheduling.** Deciding what runs where and when, under topology, power, and fairness constraints; see [[wiki/Kubernetes|Kubernetes]] · [[wiki/Slurm|Slurm]] · [[wiki/Job Scheduling|Job Scheduling]]. 7. **Serving engines.** The runtimes that turn weights into answers, with the cache management and batching that set throughput; see [[wiki/vLLM|vLLM]] · [[wiki/SGLang|SGLang]]. ## Why this layer decides the hardware market An accelerator is reached only through software, and the software requirements are large: a compiler of comparable quality, kernels for every operation a framework calls, performance tuning per generation, a collective library that handles the fabric, and tooling engineers trust. Independent analysts assess current processors and accelerators as broadly on par in capability while judging the established software ecosystem to be well ahead; the decisive element, in that assessment, is software. The effective counter-strategy has been to compete above the kernel level rather than within it. If researchers write in a framework and the framework supports several backends, the kernel library beneath becomes an implementation detail. That is why support in the major framework and in the leading open serving engines is the milestone every alternative accelerator aims at, and why open-source development in this layer has been the fastest-moving and most strategically consequential activity in the whole stack; see [[wiki/ROCm|ROCm]] · [[wiki/PyTorch|PyTorch]] · [[wiki/vLLM|vLLM]]. ## The honest summary Hardware interfaces have been substantially opened: the module, the rack, the power bus, the scale-out fabric. Software is the contested ground, and it is being contested in public, in open repositories, by engineers from competing companies. For a country that wants compute capacity sourced from more than one supply chain, this layer is where that becomes possible; see [[wiki/National Compute Capacity|National Compute Capacity]]. ## Relationships - **programs:** [[wiki/Compute and Accelerator Systems|Compute and Accelerator Systems]] · [[wiki/Accelerator|Accelerator]] - **composed of:** [[wiki/CUDA|CUDA]] · [[wiki/ROCm|ROCm]] · [[wiki/PyTorch|PyTorch]] · [[wiki/OCI Container|OCI Container]] · [[wiki/Kubernetes|Kubernetes]] · [[wiki/Slurm|Slurm]] · [[wiki/vLLM|vLLM]] · [[wiki/SGLang|SGLang]] - **determines:** [[wiki/Utilization|Utilization]] · [[wiki/Depreciation of Compute Assets|Depreciation of Compute Assets]] - **delivers:** [[wiki/Intelligence Delivery|Intelligence Delivery]] - **strategic dimension:** [[wiki/National Compute Capacity|National Compute Capacity]] · [[wiki/Supply Chain Chokepoint|Supply Chain Chokepoint]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** the seven-rung structure from driver through serving engine; host driver and in-image library version coupling; kernel library accumulation as the incumbent ecosystem's principal asset; framework-level backend support as the substitution mechanism. - **Established (datestamped):** as of October 2026, independent analysts assess processors and accelerators as broadly on par in capability while judging the established software ecosystem to be ahead and identifying software as the decisive element; the major framework and the leading open serving engines support the principal open accelerator stack. - **Strongly indicated:** open-source development in this layer as the fastest route to multi-vendor compute capacity; ecosystem activity supports it and share outcomes are still accruing. ## Sources - [Helios Is AMD's First AI System To Rival Nvidia Vera Rubin](https://www.cnbc.com/) — CNBC, October 2026; the analyst assessment of hardware parity and software lead, and open stack support for the major framework and leading inference engines.