# Accelerator Selection A facility holding several generations and several vendors of accelerator must decide which hardware runs which workload. Selection is that decision. Done well it converts a heterogeneous fleet from an operational burden into a source of price advantage and supply resilience; done poorly it is why operators are tempted to standardize on one part and accept the dependency that follows. > [!note] The one-sentence version > Match the workload to the silicon that suits it, and a warehouse of mismatched hardware starts behaving like an advantage. ## The properties that decide a match **Memory capacity.** Whether the model and its working cache fit on one device, which determines whether communication enters the critical path at all; see [[wiki/Memory Wall|Memory Wall]] and [[wiki/Scale-Up and Scale-Out|Scale-Up and Scale-Out]]. **Memory bandwidth.** The governing resource for decode, and therefore for interactive serving throughput; see [[wiki/Prefill and Decode|Prefill and Decode]]. **Arithmetic throughput and supported precisions.** The governing resource for prefill, for training, and for workloads with long prompts; see [[wiki/Quantization|Quantization]]. **Scale-up domain membership.** Whether the device participates in a large coherent fabric, which bounds tensor-parallel width and therefore which training jobs can run on it at all; see [[wiki/NVLink|NVLink]] and [[wiki/UALink|UALink]]. **Energy per unit of work.** The operating cost of serving, and at scale the figure that dominates; see [[wiki/Inference Accelerator|Inference Accelerator]]. **Software support.** Whether the workload's framework, kernels, and engine run well on that device without engineering effort. In practice this constraint eliminates more candidate placements than any physical property; see [[wiki/ROCm|ROCm]] and [[wiki/CUDA|CUDA]]. ## How selection is performed Workloads are classified by shape: prefill-heavy long-document ingestion and retrieval-augmented prompting toward compute-dense parts; decode-heavy conversational traffic toward bandwidth-rich parts; large synchronous training toward the largest available scale-up domain; development, fine-tuning, and small-model serving toward older generations and single-card systems where they run perfectly well; see [[wiki/NVIDIA RTX PRO 6000 Blackwell|NVIDIA RTX PRO 6000 Blackwell]]. The classification is then applied by the scheduler for batch work and by the routing layer for live requests; see [[wiki/Job Scheduling|Job Scheduling]] and [[wiki/Model Router|Model Router]]. ## Why this is strategically consequential A buyer able to run its workloads on more than one vendor's silicon has a second source; a buyer that cannot has a supplier. Selection is the software capability that makes the first condition real, and it is therefore the practical foundation beneath every argument in this collection about open sockets, open fabrics, and open software stacks; see [[wiki/OCP Accelerator Module|OCP Accelerator Module]] and [[wiki/Exit Cost|Exit Cost]]. At national scale the same logic holds with higher stakes. Compute capacity that can be served by several architectures is capacity less exposed to one order book, one foundry allocation, and one export decision; see [[wiki/Supply Chain Chokepoint|Supply Chain Chokepoint]], [[wiki/Compute Sovereignty|Compute Sovereignty]], and [[wiki/National Compute Capacity|National Compute Capacity]]. A scheduling and routing layer is an unlikely-looking instrument of national resilience, and it is one. ## Relationships - **part of:** [[wiki/Inference Request Routing|Inference Request Routing]] · [[wiki/Software and Orchestration|Software and Orchestration]] - **paired with:** [[wiki/Model Router|Model Router]] - **applied by:** [[wiki/Job Scheduling|Job Scheduling]] · [[wiki/Inference Serving|Inference Serving]] - **selects among:** [[wiki/Accelerator|Accelerator]] · [[wiki/Inference Accelerator|Inference Accelerator]] · [[wiki/Qualcomm Cloud AI 100 Ultra|Qualcomm Cloud AI 100 Ultra]] · [[wiki/Tenstorrent Galaxy Blackhole|Tenstorrent Galaxy Blackhole]] - **enabled by:** [[wiki/ROCm|ROCm]] · [[wiki/OCP Accelerator Module|OCP Accelerator Module]] - **strategic dimension:** [[wiki/Compute Sovereignty|Compute Sovereignty]] · [[wiki/Exit Cost|Exit Cost]] · [[wiki/National Compute Capacity|National Compute Capacity]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** the deciding properties of memory capacity, memory bandwidth, arithmetic throughput and precision support, scale-up domain membership, energy per unit of work, and software support; workload classification by prefill-heavy and decode-heavy shape and its application through schedulers and routing layers; software support eliminating more candidate placements in practice than physical properties. - **Established (datestamped):** as of October 2026, managed inference services in the industry provide automated hardware selection governed by model requirements, performance, and total cost considerations. - **Strategic assessment:** the judgement that a selection layer is a practical instrument of supply resilience at both buyer and national scale is an argued position rather than a measured finding. ## Sources - Managed inference service documentation and heterogeneous cluster scheduling practice, as current October 2026.