# Utilization Utilization is the fraction of a facility's capacity that becomes useful work. For capital this concentrated and depreciating this quickly, it is the metric that decides whether the investment pays, and it is the metric most often reported in a flattering form. > [!note] The one-sentence version > Of everything the building could compute this month, how much did it actually compute; every percentage point is money. ## The measures, from flattering to honest - **Allocation rate.** The share of accelerators assigned to a job. Easy to report and nearly meaningless, since an assigned accelerator may be waiting on data or stalled in a collective. - **Device busy time.** The share of time the device reports activity. Better, and still generous, because a device spinning through a communication wait registers as busy. - **Model utilization of arithmetic capacity.** Achieved arithmetic throughput against the device's theoretical peak, for the specific computation being run. This is the honest engineering figure, and well-optimized large training runs achieve a substantial fraction of peak rather than anything close to all of it; the gap is communication, memory stalls, and the parts of a model that do not map onto peak-rate arithmetic. - **Goodput.** Work that survived to be useful, excluding computation lost to failures and restarts. The only figure that corresponds to value, and the hardest to produce. Honest reporting states which measure is meant. A facility quoting high utilization without saying which number it is has said very little. ## Where it leaks **Queueing and fragmentation.** Devices idle because no queued job fits the free shape; see [[wiki/Job Scheduling|Job Scheduling]]. **Data starvation.** The input pipeline cannot keep pace, so accelerators wait on storage or host preprocessing; see [[wiki/Storage Tiering|Storage Tiering]] and [[wiki/Host CPU|Host CPU]]. **Communication.** Time spent in collectives rather than computation, worsened by poor placement; see [[wiki/Collective Communication|Collective Communication]] and [[wiki/Network Fabric Topology|Network Fabric Topology]]. **Stragglers.** One slow device pacing a whole group; see [[wiki/Site Reliability Engineering|Site Reliability Engineering]]. **Thermal throttling.** Devices reducing clocks to stay within limits, which is a facilities failure appearing as a performance one; see [[wiki/Data Center Cooling|Data Center Cooling]]. **Failures and restarts.** Work lost between the last checkpoint and the fault; see [[wiki/Checkpointing|Checkpointing]]. **Idle serving capacity.** Workers held warm against peak demand, which is a deliberate and sometimes substantial cost of meeting a latency commitment; see [[wiki/Inference Serving|Inference Serving]]. ## Why it is the central operating concern The capital in a dense facility is dominated by equipment on a short depreciation clock, so idle time is not merely lost opportunity but consumed asset life; see [[wiki/Capital Intensity|Capital Intensity]] and [[wiki/Depreciation of Compute Assets|Depreciation of Compute Assets]]. The practical consequence is that software and operations improvements, better scheduling, better data pipelines, better placement, better straggler detection, more even cooling, routinely return more than a hardware purchase of equal cost. That is an encouraging fact: much of the headroom in the American buildout lies in running what has been built well, which is work that can begin immediately and requires no supply chain at all. ## Relationships - **leaked through:** [[wiki/Job Scheduling|Job Scheduling]] · [[wiki/Collective Communication|Collective Communication]] · [[wiki/Storage Tiering|Storage Tiering]] · [[wiki/Data Center Cooling|Data Center Cooling]] · [[wiki/Checkpointing|Checkpointing]] - **determines return on:** [[wiki/Capital Intensity|Capital Intensity]] · [[wiki/Depreciation of Compute Assets|Depreciation of Compute Assets]] - **measured through:** [[wiki/Observability|Observability]] - **managed by:** [[wiki/Site Reliability Engineering|Site Reliability Engineering]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** the distinction between allocation rate, device busy time, achieved arithmetic utilization against peak, and goodput; communication, memory stalls, and non-peak-rate model components as the gap between busy time and arithmetic utilization; the leak categories of queueing and fragmentation, data starvation, communication, stragglers, thermal throttling, failure and restart, and warm idle serving capacity. - **Established:** well-optimized large training runs achieving a substantial fraction rather than the whole of theoretical peak arithmetic throughput. - **Strongly indicated:** operations and software improvements returning more than equal-cost hardware purchases at typical utilization levels; the reasoning is clear and published cross-facility measurement is limited. ## Sources - Large-scale training efficiency literature and cluster operations practice, as current October 2026.