# Job Scheduling A cluster has fewer accelerators than people want and more constraints than anyone can hold in mind. The scheduler decides which work runs, on which devices, in which order, subject to every one of those constraints. It is the component where a facility's physical reality, its topology, its power envelope, its failure domains, becomes an input to software decisions, and it determines what fraction of an enormous capital investment is actually converted into work. > [!note] The one-sentence version > Given a queue of jobs and a building full of constraints, decide what runs where; everything the facility is worth passes through that decision. ## The constraints a scheduler must respect - **All-or-nothing allocation.** A synchronous distributed job needs every worker at once; partial placement wastes the accelerators already held; see [[wiki/Collective Communication|Collective Communication]]. - **Topology.** Workers should sit inside one scale-up domain where possible, then one leaf, then one spine group, because placement determines achieved bandwidth; see [[wiki/Scale-Up and Scale-Out|Scale-Up and Scale-Out]] and [[wiki/Network Fabric Topology|Network Fabric Topology]]. - **Device heterogeneity.** Different accelerator generations and types have different memory capacities and performance; a job must land on hardware that suits it. - **Power envelope.** A hall has a megawatt limit, and simultaneous peak draw across many racks can exceed what was provisioned; schedulers increasingly treat power as a schedulable resource, which is also the mechanism for grid participation; see [[wiki/Demand Response|Demand Response]]. - **Failure domains.** Spreading replicas across power and cooling domains for serving, and concentrating training workers for bandwidth, are opposite requirements the scheduler must distinguish; see [[wiki/Failure Domain|Failure Domain]]. - **Fairness and priority.** Quotas, entitlements, borrowing between teams, and preemption of low-priority work. - **Data locality.** Placing work near its staged data, or triggering the staging in advance; see [[wiki/Storage Tiering|Storage Tiering]]. ## The scheduling techniques **Gang scheduling** places a whole group atomically. **Backfill** fills gaps ahead of a large queued job with short work that finishes in time, which is the main tool for keeping utilization high in a mixed queue. **Preemption** reclaims resources for higher-priority work, and is only humane when the preempted job can resume, which is why [[wiki/Checkpointing|checkpointing]] and scheduling are a single design question. **Bin packing** consolidates small work onto fewer nodes to leave contiguous space for large jobs; a cluster that packs carelessly becomes unable to run a large job despite having enough free devices, which is fragmentation and is the characteristic failure of naive allocation. **Reservations** hold capacity for known future needs. **Elastic jobs** grow and shrink their worker count, which suits some training approaches and makes a queue far more fluid. ## Why this is where utilization is won The gap between a cluster's theoretical capacity and its delivered work is largely made of scheduling losses: fragmentation, queueing delay, jobs placed across the wrong boundary, stragglers admitted by a missing health check, and devices idle between jobs while data stages. Each is a software problem with a known remedy, and together they are commonly worth more than any hardware upgrade available at the same cost. For capital this concentrated, the scheduler is the highest-leverage software in the building; see [[wiki/Utilization|Utilization]] and [[wiki/Capital Intensity|Capital Intensity]]. ## Relationships - **implemented by:** [[wiki/Slurm|Slurm]] · [[wiki/Kubernetes|Kubernetes]] - **respects:** [[wiki/Network Fabric Topology|Network Fabric Topology]] · [[wiki/Failure Domain|Failure Domain]] · [[wiki/Storage Tiering|Storage Tiering]] - **depends on:** [[wiki/Checkpointing|Checkpointing]] - **determines:** [[wiki/Utilization|Utilization]] · [[wiki/Capital Intensity|Capital Intensity]] returns - **enables:** [[wiki/Demand Response|Demand Response]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** gang scheduling, backfill, preemption, bin packing, reservations, and elastic jobs as standard techniques; fragmentation preventing large job placement despite sufficient free devices; topology-aware placement affecting achieved collective bandwidth; the opposing placement requirements of serving replicas and training workers. - **Strongly indicated:** power treated as a schedulable resource in production clusters; operators describe the capability and sustained operating records are still short. - **Strongly indicated:** scheduling losses exceeding available hardware upgrades in value at equal cost; the reasoning is clear and published cross-facility measurement is limited. ## Sources - Cluster scheduling literature and production workload manager documentation, as current October 2026.