# Gradient Descent **Domain:** Optimization / Machine Learning **Doc Type:** Canonical Concept Node **Maturity:** Foundational ## Definition **Gradient descent** is a family of iterative optimization methods that adjust parameters in a direction expected to reduce an objective function, using local derivative information or an estimate of it. ## Evolutionary Relation [[articles/The Evolutionary Roots of Silicon Valley|The Evolutionary Roots of Silicon Valley]] calls gradient descent hill-climbing on a fitness landscape. The structural analogy is useful: both move through a space under differential evaluation. The mechanisms differ. Gradient descent follows local slope in a differentiable parameter space; natural selection acts on inherited population variation without calculating a gradient. This distinction matters because gradient descent can be highly efficient while remaining sensitive to representation, initialization, local geometry and the chosen loss. It does not discover purpose; it follows the evaluative surface designers and data construct. ## Governance Context Optimization hides normative choices inside differentiable proxies. If a harmed population is absent from the objective, the gradient has no reason to protect it. ## Key Insight **Gradient descent supplies direction, not justification: it can tell a system how to reduce a loss without establishing that the loss deserves to govern.** ## Sources / Provenance - Léon Bottou, Frank Curtis and Jorge Nocedal, “Optimization Methods for Large-Scale Machine Learning,” _SIAM Review_ 60 (2018). - Nexus synthesis: [[articles/The Evolutionary Roots of Silicon Valley|The Evolutionary Roots of Silicon Valley]]. ## See Also [[wiki/Objective Function|Objective Function]], [[wiki/Selection Gradient|Selection Gradient]], [[wiki/Machine Learning|Machine Learning]], [[wiki/Natural Selection|Natural Selection]] ## Simple Reminders, Quotations, and Thoughts > Artificial neural networks are trained through gradient descent, yet biology has not revealed anything clearly equivalent in the brain. Detailed neural imaging may uncover the actual algorithm that changes synaptic weights—and that discovery could make machine learning ten, a hundred, a thousand, or even a million times more efficient. > **— Adapted from Dave Blundin**, *MOONSHOTS Live, October 2026* [[reminders/Biotechnology/Biology May Reveal a Learning Algorithm Far Beyond Gradient Descent by Dave Blundin|Biology May Reveal a Learning Algorithm Far Beyond Gradient Descent by Dave Blundin]]