# RDMA Remote direct memory access is the ability of one machine to read or write a region of another machine's memory without the remote processor or operating system taking part. The network adapter does the work: it receives the request, performs the memory access by direct memory access, and completes. The remote processor may be busy with something else entirely and never learns that the transfer occurred. > [!note] The one-sentence version > Put the data in the other machine's memory yourself, and leave its processor out of it. ## Why it exists Conventional networking is expensive in processor time. Sending data means copying it from the application's buffer into the kernel, through the protocol stack, into the adapter; receiving reverses the process, with an interrupt, a context switch, and a copy at every stage. At a few gigabits per second that cost was acceptable. At hundreds of gigabits per second it would consume a large fraction of the processors in the machine simply to move data, which is unaffordable when those processors are meant to be feeding accelerators; see [[wiki/Host CPU|Host CPU]]. Direct access removes all of it: no copies, no kernel transition on the data path, no interrupt per message, and no processor involvement at the receiver. ## Prerequisites - **Memory registration:** the application tells the adapter which memory regions it may access, and the operating system pins those pages so they cannot move; this is the one-time setup cost that makes the fast path possible. - **Queue pair:** send and receive queues the application posts work to directly in user space, bypassing the kernel entirely during operation. - **Zero copy:** data moves from the application's buffer to the wire with no intermediate copies. - **One-sided and two-sided operations:** a write or read that the remote side need not participate in, against a send and receive pair that both ends post to. - **Completion queue:** where the adapter reports finished work, polled or signalled by event. - **Protection domain and keys:** the mechanism restricting which remote peers may touch which regions, which is the security boundary of the whole arrangement. ## The transports **Native fabric.** Direct access is the native model of a purpose-built cluster fabric; see [[wiki/InfiniBand|InfiniBand]]. **Over Ethernet.** The same operations are carried over Ethernet, which requires the Ethernet fabric to behave: priority-based flow control and explicit congestion notification are configured so that the lossless conditions direct access assumes actually hold. Making that work reliably at scale is a genuine engineering discipline, and getting the configuration wrong produces the characteristic failure of the technology, which is a fabric that performs beautifully in testing and collapses under a specific congestion pattern in production. Consortium work has been directed precisely at making this robust without hand-tuning; see [[wiki/Ultra Ethernet|Ultra Ethernet]] and [[wiki/Congestion Control|Congestion Control]]. ## What depends on it Nearly everything above the fabric. Collective operations across thousands of accelerators; see [[wiki/Collective Communication|Collective Communication]]. Parallel storage delivering training data at full rate; see [[wiki/Parallel File System|Parallel File System]]. Checkpoint writes and restores; see [[wiki/Checkpointing|Checkpointing]]. Cache transfer between serving pools in disaggregated inference; see [[wiki/Prefill and Decode|Prefill and Decode]]. Accelerators can also be the direct source or destination, so data moves between a remote machine and accelerator memory without touching host memory at all, which is the mechanism behind the highest-throughput data paths in the building. ## Relationships - **transported by:** [[wiki/InfiniBand|InfiniBand]] · [[wiki/Ultra Ethernet|Ultra Ethernet]] · [[wiki/DPU and SmartNIC|DPU and SmartNIC]] - **enables:** [[wiki/Collective Communication|Collective Communication]] · [[wiki/Parallel File System|Parallel File System]] · [[wiki/NVMe over Fabrics|NVMe over Fabrics]] · [[wiki/Checkpointing|Checkpointing]] - **depends on:** [[wiki/Congestion Control|Congestion Control]] - **relieves:** [[wiki/Host CPU|Host CPU]] - **source articles:** [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] · [[articles/The Gigawatt Is the New Measure of War|The Gigawatt Is the New Measure of War]] - **collection route:** [[collections/American Infrastructure|American Infrastructure]] · [[collections/American|American Collection]] ## Evidence posture - **Established:** the processor cost of conventional networking at high rates; memory registration and pinning; user-space queue pairs with kernel bypass; zero-copy transfer; one-sided and two-sided operations; completion queues; protection domains and keys as the access-control mechanism; direct transfer into accelerator memory without host staging. - **Established:** the requirement for lossless fabric behavior when carrying direct access over Ethernet, achieved through priority flow control and explicit congestion notification, and consortium work directed at making that robust at scale. - **Strongly indicated:** misconfigured congestion control as the characteristic production failure of Ethernet-carried direct access; operator experience is consistent and consolidated incident data is not public. ## Sources - InfiniBand Trade Association and Ultra Ethernet Consortium specifications, as current October 2026. - High-performance networking practice for direct memory access transports, as of October 2026.