# Checkpointing **Domain:** Distributed Computing / Storage / Fault Tolerance **Doc Type:** Technical Concept Node **Maturity:** Developed ## Definition **Checkpointing** is the periodic persistence of sufficient computational state to resume useful work after interruption or failure. ## Governing Variable Checkpointing governs recoverable progress: how much completed computation survives a fault and how quickly execution can restart. ## Material and Computational Manifestation Large training jobs may preserve model parameters, optimizer state, scheduler state and data position across distributed storage. The checkpoint path therefore joins accelerator memory, network bandwidth, storage throughput and consistency logic. ## Relation to Governance At scale, component failure is expected. Checkpoint policy decides which state counts as durable, how often the system pays the cost of persistence and which prior state can authoritatively restore the computation. ## Constitutional Question and Failure Modes Checkpoints create continuity but can preserve corrupted or unauthorized state. Failure modes include inconsistent snapshots, storage congestion, excessive recovery time, silent incompleteness and retention policies that make rollback impossible or expose sensitive model state. ## Related Ontology [[wiki/Fault-Tolerant Computing|Fault-Tolerant Computing]] · [[wiki/NVMe over Fabrics|NVMe over Fabrics]] · [[wiki/Systems of Record|Systems of Record]] · [[wiki/AI Factory|AI Factory]] · [[wiki/Continuous Reconciliation|Continuous Reconciliation]]