# Network Fabric Topology Topology is the pattern in which switches and endpoints are wired. It decides how much bandwidth any two endpoints can actually use, how the network behaves when many pairs communicate at once, and what the software above must know. In a training cluster, topology is not an implementation detail; it is part of the machine's architecture. > [!note] The one-sentence version > The shape of the cabling sets which conversations can happen at full speed, and the software has to be written to match. ## The standard shapes **Leaf and spine.** Endpoints connect to leaf switches; every leaf connects to every spine switch; traffic between endpoints on different leaves crosses exactly one spine. Every pair is therefore the same distance apart, and capacity grows by adding spines. This two-tier arrangement is the backbone of modern data center networking. **Fat tree.** The multi-tier generalization, with uplink capacity at each level sized so that the aggregate bandwidth does not narrow as traffic moves upward. A fully provisioned fat tree is **non-blocking**: any permutation of endpoint pairs can communicate simultaneously at full rate. This is what training clusters aim for and what makes them expensive, since non-blocking means roughly as much switching and optics above the endpoints as the endpoints themselves require. **Oversubscription.** Providing less uplink capacity than downlink, expressed as a ratio. A three-to-one oversubscribed leaf is cheaper and perfectly reasonable for workloads whose traffic is mostly local. It is a poor choice for synchronous collectives, which produce exactly the all-pairs pattern oversubscription penalizes; see [[wiki/Collective Communication|Collective Communication]]. **Rail-optimized.** The design specific to accelerator clusters. Each accelerator in a node has its own network interface, and all the interfaces holding the same position across nodes connect to the same switch, forming a rail. The accelerator in position three of every node reaches the accelerator in position three of every other node across one dedicated rail. This matches collective traffic patterns precisely, since a ring or hierarchical all-reduce principally communicates between corresponding ranks, and it keeps that traffic on a single switch hop. The arrangement is one of the more satisfying pieces of design in the whole stack: the wiring was reshaped to fit the mathematics. **Dragonfly and torus.** Topologies from high-performance computing that trade uniformity for fewer long cables and lower cost at very large scale, requiring more topology-aware software. ## Oversubscription, locality and scheduling Topology and the scheduler must agree. A job placed across racks that share a leaf performs differently from one scattered across a hall, and a scheduler that ignores topology will produce inexplicable variation in training throughput from one run to the next. Modern schedulers therefore allocate by topology, preferring contiguous blocks of accelerators within one scale-up domain, then within one leaf, then within one spine group; see [[wiki/Job Scheduling|Job Scheduling]]. The consequence for capacity planning is that a cluster's usable size for one large job is smaller than its total device count, because placement must be contiguous to perform. ## What it costs Non-blocking fabric at these port counts is a substantial share of a cluster's capital and power: switch silicon, transceivers, and fiber in quantities comparable to the compute. This is why [[wiki/Data Center Optics|optics]] economics matter so much, why in-rack copper is preserved wherever reach allows, and why enlarging the scale-up domain is attractive, since every accelerator pair moved inside the domain is a pair removed from the expensive network. ## Relationships - **carries:** [[wiki/Collective Communication|Collective Communication]] · [[wiki/RDMA|RDMA]] - **implemented with:** [[wiki/InfiniBand|InfiniBand]] · [[wiki/Ultra Ethernet|Ultra Ethernet]] · [[wiki/Data Center Optics|Data Center Optics]] · [[wiki/Structured Cabling|Structured Cabling]] - **constrains:** [[wiki/Job Scheduling|Job Scheduling]] · [[wiki/Utilization|Utilization]] - **traded against:** [[wiki/Scale-Up and Scale-Out|Scale-Up and Scale-Out]] domain size · [[wiki/Capital Intensity|Capital Intensity]] - **managed through:** [[wiki/Congestion Control|Congestion Control]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** leaf-spine and fat-tree construction; non-blocking provisioning and its cost; oversubscription ratios and their unsuitability for synchronous collectives; rail-optimized designs connecting corresponding accelerator positions across nodes to dedicated switches; dragonfly and torus topologies from high-performance computing; topology-aware scheduling and its effect on training throughput consistency. - **Established:** switching, transceivers, and fiber as a substantial share of cluster capital and power at non-blocking provisioning. - **Strongly indicated:** contiguous topology-aware placement materially improving large-job performance; scheduler design and operator practice converge on it and published measurements vary by workload. ## Sources - Data center and high-performance cluster network design practice, including rail-optimized accelerator fabrics, as current October 2026.