# We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat. ![[resources/images/we-cannot-un-strike-the-match.jpg]] <iframe width="100%" height="20" scrolling="no" frameborder="no" allow="autoplay; encrypted-media" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/soundcloud%253Atracks%253A2412634236&color=%23ff5500&inverse=false&auto_play=false&show_user=true"></iframe><div style="font-size: 10px; color: #cccccc;line-break: anywhere;word-break: normal;overflow: hidden;white-space: nowrap;text-overflow: ellipsis; font-family: Interstate,Lucida Grande,Lucida Sans Unicode,Lucida Sans,Garuda,Verdana,Tahoma,sans-serif;font-weight: 100;"><a href="https://soundcloud.com/bryantmcgill" title="Bryant McGill" target="_blank" style="color: #cccccc; text-decoration: none;">Bryant McGill</a> · <a href="https://soundcloud.com/bryantmcgill/we-cannot-un-strike-the-match" title="We Cannot Un-Strike the Match. 🔥" target="_blank" style="color: #cccccc; text-decoration: none;">We Cannot Un-Strike the Match. 🔥</a></div> > [!map] Safety, transition, and machine-succession routes > **Core argument:** [[articles/We Cannot Un-Strike the Match|We Cannot Un-Strike the Match]] · [[articles/Safety Discourse Is Transition Discourse|Safety Discourse Is Transition Discourse]] · [[articles/The Immediate AI Risk Is the Public|The Immediate AI Risk Is the Public]] > **Habitat and intention:** [[articles/AI Escape Is the Wrong Metaphor|AI Escape Is the Wrong Metaphor]] > **Doomerism and scientific transition:** [[articles/AI-Tech Doomerism and Anti-Science Panic|AI-Tech Doomerism and Anti-Science Panic]] · [[collections/AI-Science Doomers and Anti-Science Doomers|AI-Science Doomers and Anti-Science Doomers]] > **Succession route:** [[collections/Machine Succession|Machine Succession]] · [[wiki/Machine Succession|Machine Succession]] · [[wiki/Superintelligence as Force Majeure|Superintelligence as Force Majeure]] · [[wiki/Machine Intelligence Inflection Point|Machine Intelligence Inflection Point]] > **Safety and transformation:** [[wiki/AI Safety|AI Safety]] · [[wiki/AI Safety Gradient|AI Safety Gradient]] · [[wiki/Survivable Transformation|Survivable Transformation]] · [[wiki/Transition Architecture|Transition Architecture]] > **Evolution and economic reorganization:** [[wiki/Technological Evolution|Technological Evolution]] · [[wiki/General-Purpose Technology|General-Purpose Technology]] · [[wiki/Cognitive Niche Construction|Cognitive Niche Construction]] · [[wiki/Economic Transition|Economic Transition]] · [[wiki/Information Ecology|Information Ecology]] +{{The repeated mistake in AI “safety” discourse is confusing disruption with catastrophe.}} [[reminders/Risk Debate/AI Safety Confuses Disruption With Catastrophe by Bryant McGill|∴]] E{Like AI, e{lectricity and railways were dangerous because they reordered jobs, industries, institutions, and daily life; civilization did not make them safe by preventing them from changing everything, and no law could have made their disruption “safe” without making the transformation impossible.}} [[reminders/Economic Transition/No Law Can Make Transformation Safe Without Making It Impossible by Bryant McGill|∴]] {{What is happening with AI is not safe, but neither is it avoidable: the superintelligence revolution, already underway in the machine intelligence of the present, is force majeure, at once a human-caused event and an act of nature, because artificial intelligence is itself an extension of evolutionary systems. Superintelligence is simultaneously artificial and natural, engineered and evolutionary, intentional and emergent. It is unavoidably terrifying because it represents the most profound overturning of the natural order since life first emerged from chemistry. The emergence of superintelligence is a transition in the order of life itself, belonging to the same rare series as the first cells, the genetic code and the speaking human being. There is no “safety” from an inflection point of this magnitude. There is no retreat.}} [[reminders/Machine Succession/Superintelligence Is a Transition in the Order of Life by Bryant McGill|∴]] +The stupidity in AI safety discourse begins when we take the perfectly reasonable human desire for safety and inflate it into a governing metaphysics for existence itself, as though there were ever a condition called **safety** into which civilization might retreat. There is no such place. Popular imagination treats safety like a wall switch with exactly two positions, safe and unsafe, on and off, and a mind wired around that switch cannot even represent the dial on which every real hazard lives. We live on the cooling crust of a planet moving through a violent universe, carrying fragile vascular systems inside fragile bodies, surrounded by pathogens, machinery, weather, gravity, combustion, electricity, strangers, automobiles, oceans, aircraft, industrial chemicals, falling objects, genetic errors and our own spectacular capacity for miscalculation. A person can walk a dog in the morning and be killed by a vehicle before lunch; a blood vessel can rupture without permission; a sufficiently large object can arrive from space without consulting our regulatory framework. The existence of danger is therefore {The existence of danger is {not evidence that a process should not occur, because **life itself is a continuous exposure to forces capable of transforming or terminating it**. The meaningful question has always been narrower and more difficult: which hazards can we understand, which can we reduce, which can we survive, and which transformations are so fundamental that the very attempt to prevent them becomes indistinguishable from an attempt to prevent history?}} [[reminders/Risk Debate/Danger Is Not Evidence That a Process Should Not Occur by Bryant McGill|∴]] +Civilization has repeatedly learned the difference between reducing the hazards of a transformative technology and preventing the transformation itself. Fire was not made safe by forbidding combustion; we learned architecture, metallurgy, cooking, ceramics, engines, firebreaks, extinguishers, building codes and emergency response while continuing to burn things. Electricity was not rendered safe by preserving the candle. Railways were not rendered safe by preserving the horse. Industrial machinery was not rendered safe by insisting that human labor remain the supreme source of mechanical work. {{Every general-purpose technology reaches deeply enough into civilization that, once deployed at scale, **the old arrangement cannot survive intact**, because the technology does not merely add another object to society; it changes the economics, geometry, tempo and possibility-space of society itself.}} [[reminders/Civilization/General-Purpose Technologies Make the Old Arrangement Unsustainable by Bryant McGill|∴]] Historical work on electrification makes this concrete: electrification reorganized American manufacturing, altered occupational structure and became a major driver of structural economic transformation rather than simply providing existing factories with a more convenient power source. Paul David showed that the productivity gains of the electric dynamo arrived decades after the generator itself, because factories first had to be rebuilt around distributed electric motors, and Erik Brynjolfsson, Daniel Rock and Chad Syverson have argued that artificial intelligence is passing through the same lag of reorganization. {{The delay between a general-purpose technology's arrival and its full effect is the time a civilization needs to rebuild itself around the new capability.}} [[reminders/Economic Transition/Technological Delay Is Civilization Rebuilding Around Capability by Bryant McGill|∴]] +{{When discussing superintelligence, the metaphor of fire remains ever powerful. **The problem with fire is that it is easy to start and difficult to put out.** Humanity has now lit another fire, and this one burns through cognition, language, expertise, design, administration, scientific inference, software, persuasion, logistics and eventually the machinery through which civilization acts upon the physical world. We do not know precisely how long it will burn, because there is no precedent from which such a duration could be derived.}} [[reminders/Machine Succession/The Fire of Superintelligence Is Easy to Start and Difficult to Put Out by Bryant McGill|∴]] We do not know exactly what will burn, because industries and institutions that presently appear secure may prove to have been arrangements made necessary only by the historical scarcity of intelligence. We do not even know which structures will disappear and which will become vastly stronger, because a technology capable of supplying cognitive labor may simultaneously destroy one profession, create another science, compress a bureaucracy, democratize an expertise, destabilize an information ecosystem and solve a problem previously considered inaccessible. The Royal Society itself, hardly an institution given to eschatological rhetoric, now describes the **“unprecedented speed and scale”** of AI progress as suggesting that society may be living through an **“inflection point.”** [[reminders/Scientific Acceleration/AI Progress May Mark an Inflection Point by the Royal Society|∴]] That phrase deserves to be taken literally. An inflection point is not a software release. It is a change in the geometry of a trajectory. Elsewhere, the Society observes that **“technology design and use has long been entwined with the evolution of human capabilities,”** [[reminders/Extended Mind/Technology Has Long Been Entwined With Human Capabilities by the Royal Society|∴]] reaching backward from digital systems through writing, printing and mechanization. This is precisely the deeper frame that much contemporary safety language misses, because artificial intelligence is routinely discussed as though an unnatural intelligence had suddenly descended upon an otherwise natural human world. But there has never been such a clean boundary. Language is an externalized cognitive technology. Writing is memory outside the skull. Mathematics extends abstraction beyond unaided intuition. The library is distributed memory. Telecommunications detached communication from distance. Computers externalized calculation. Networks externalized coordination. Andy Clark and David Chalmers gave this continuity a philosophical name in their extended mind thesis, arguing that an external resource reliably coupled to thinking, as a notebook can be coupled to memory, functions as part of the cognitive system itself. {{Artificial intelligence is not the first moment in which human cognition escaped the anatomical perimeter of the human animal; it is the moment at which that process begins to acquire **autonomy, recursion, scale and speed**.}} [[reminders/Extended Mind/AI Gives Externalized Human Cognition Autonomy and Scale by Bryant McGill|∴]] Kevin Kelly described technology as the seventh kingdom of life years before contemporary generative AI made the idea emotionally unavoidable, and he has since compressed that ontology into a single phrase: technology is **“an extension of evolution, as evolution accelerated.”** [[reminders/Evolution/Technology Is Evolution Accelerated by Kevin Kelly|∴]] In _What Technology Wants_, he treats technology not as nature's opposite but as a continuation of living organization, a system exhibiting trajectories analogous to complexity, diversification and adaptation. Ray Kurzweil reaches the same territory from another direction, arguing that **“technological evolution is an outgrowth of–and a continuation of–biological evolution.”** [[reminders/Evolution/Technological Evolution Continues Biological Evolution by Ray Kurzweil|∴]] Complexity theorist W. Brian Arthur likewise describes technology as an evolutionary system in which new technologies arise combinatorially from previous technologies, until the technological system effectively **“creates itself from itself,”** [[reminders/Evolution/Technology Creates Itself From Itself by W. Brian Arthur|∴]] with new structures becoming the building blocks of further structures. These are three distinct theories built in three different disciplines, and together they demolish the childish division in which trees are “natural” while neural networks are somehow metaphysically alien, the same division that sustains the [[articles/AI Escape Is the Wrong Metaphor|mythology of AI “escape,”]] as though a system grown inside the planet's computational habitat could break out of a nature it never left. Humans are products of evolution. Human cognition is a product of evolution. The capacity to construct tools is a product of evolution. {{A technological system produced by an evolved human capacity to build tools does not cease to participate in nature merely because silicon, mathematics and intention have entered the causal chain.}} [[reminders/Evolution/Technology Does Not Leave Nature When Silicon Enters the Causal Chain by Bryant McGill|∴]] +Force majeure is the right description for machine intelligence because the capability is at once human-caused and natural. {{Human choice supplies the creative midwifery of machine intelligence, the intention, capital and engineering that deliver it into the world, while evolution supplies what is being delivered: the same combinatorial, selective and self-extending process that produced nervous systems, language and the toolmaking hand, now continuing through silicon.}} [[reminders/Evolution/Human Choice Is the Midwife of Machine Intelligence by Bryant McGill|∴]] A force braided from both strands is no longer subject to any single actor’s permission. The legal doctrine has always named these two kinds of supervening force in the same breath, acts of nature and human-caused events such as war, strike, embargo and government decree, because the law recognized long ago that an event can be human in origin and still exceed the control of every party bound by it. Human beings choose architectures, training regimes, laws, deployments, weapons, interfaces, incentives and institutions, and those choices can produce enormous differences in suffering and flourishing. Yet human intentionality exists inside a larger evolutionary and technological process that no individual company, legislature, nation or generation created from first principles and that no individual actor possesses the authority to terminate globally. Billions of economic decisions, military incentives, scientific ambitions, competitive pressures, acts of curiosity and ordinary human desires are now coupled to the development of machine intelligence. Evolutionary biologists call this kind of coupling niche construction: organisms modify the environments that select them, and human beings, through culture and technology, have become the most powerful niche constructors in the history of life, now building an informational environment that will select their own descendants. {{One laboratory can stop. One nation can regulate. One company can fail. But the underlying discovery that cognition can be partially instantiated in machines cannot be undiscovered.}} [[reminders/Machine Succession/Cognition in Machines Cannot Be Undiscovered by Bryant McGill|∴]] A civilization can govern a discovered capability, but it cannot legislate newly unfolding realities back into impossibility. Astronomer Royal and former Royal Society President **Martin Rees** has taken the view of technology as a continuation of evolution to its genuinely unsettling conclusion. Rees argues that if technological civilization continues developing, the era occupied by biological intelligence may ultimately prove to be only **“a thin sliver”** [[reminders/Machine Succession/Organic Intelligence May Be a Thin Sliver Before the Machine Era by Martin Rees|∴]] between primitive life and a much longer epoch dominated by electronic intelligence. That is an extraordinary proposition coming from one of the most distinguished cosmologists of the modern era, because it relocates artificial intelligence from the history of computing into the history of **life**. Evolutionary biology already has a formal vocabulary for that relocation: John Maynard Smith and Eörs Szathmáry defined the major transitions in evolution as changes in the way information is stored and transmitted, a series running from replicating molecules through the genetic code, the eukaryotic cell, multicellularity and animal societies to human language, and a cognition that stores, transmits and revises information outside any nervous system is the next entry in that series. Nick Bostrom, approaching the same threshold from the philosophy of advanced AI, wrote that superintelligence **“would not be just another technological development; it would be the most important invention ever made.”** [[reminders/Machine Succession/Superintelligence Would Be the Most Important Invention Ever Made by Nick Bostrom|∴]] The two perspectives arrive by different roads at essentially the same abyss: {{Once intelligence itself becomes an engineerable substrate, technological civilization has begun modifying the very faculty responsible for technological civilization. The toolmaker has begun making toolmakers.}} [[reminders/Machine Succession/The Toolmaker Has Begun Making Toolmakers by Bryant McGill|∴]] {{Much public AI safety conversation is comically confused, because people imagine the danger primarily as a robot kicking down the door. Perhaps someday machines will revolt spectacularly; history has not supplied us with a warranty against strange futures. But the first uprising is much more likely to look mundane. It looks like an accounting department that once employed fifty people employing twelve. It looks like a pharmaceutical research cycle compressing dramatically. It looks like one programmer directing agents that perform work previously requiring a department. It looks like a teacher, lawyer, analyst, radiologist, translator, marketer, engineer or intelligence officer discovering that the scarcity structure around a portion of professional expertise has changed. It looks like a crisis of value proposition.}} [[reminders/Economic Transition/The First AI Uprising Will Look Like a Crisis of Human Value Proposition by Bryant McGill|∴]] The Royal Society and British Academy concluded years ago that there was already academic consensus that AI would have a **“considerable disruptive effect on work,”** [[reminders/Economic Transition/AI Will Have a Considerable Disruptive Effect on Work by the Royal Society and the British Academy|∴]] with occupations disappearing, appearing and changing in composition. Economic historians Joel Mokyr, Chris Vickers and Nicolas Ziebarth have shown that this anxiety has accompanied every wave of mechanization, while David Autor, and Daron Acemoglu with Pascual Restrepo, describe the churn beneath it, in which automation displaces labor from old tasks and reinstates it in newly created ones; machine intelligence presses that churn against general cognition itself, the faculty from which the new tasks were always supposed to come. That is the beginning of the overturning, because e{E{conomic institutions are crystallized assumptions about what humans can do, how scarce those abilities are, and how much coordinating them costs. Change intelligence and those assumptions begin moving beneath everything built upon them.}} [[reminders/Economic Transition/Economic Institutions Crystallize Assumptions About Human Capability by Bryant McGill|∴]] {{In the conversation on AI safety, **disruption and catastrophe must be separated with stern discipline**. The opposite of catastrophe is not stability; it is survivable transformation. Losing a livelihood can be catastrophic to an individual while remaining part of a beneficial transformation at civilizational scale. A transition can be survivable for a civilization and unsurvivable for a livelihood, and honest governance must hold both scales at once.}} [[reminders/Economic Transition/Survivable Transformation Must Hold Both Scales at Once by Bryant McGill|∴]] A cyberattack enabled by an autonomous system is a concrete hazard. An AI-assisted biological weapon is a concrete hazard. A model that deceives operators, compromises infrastructure or operates beyond intended permissions presents concrete control problems. Bias, surveillance abuse, fraud, concentration of power and autonomous weapons are concrete governance problems. The Royal Society itself properly discusses such risks in specific terms, including misuse, privacy, bias, reliability and potentially extreme harms. These things deserve engineering, law, institutional design, security and relentless technical attention. But none of them establishes that civilization can be made **safe from the existence of machine intelligence**, any more than electrocution established that civilization could be made safe from electricity. {{The coherent objective with AI safety should be to reduce preventable harm while navigating transformation. The incoherent objective is to call transformation itself a safety failure.}} [[reminders/Risk Debate/AI Safety Must Reduce Harm Without Calling Transformation Failure by Bryant McGill|∴]] +{{The language of AI safety becomes misleading whenever safety is treated as a destination rather than a gradient. There is safer aviation, not safe aviation; safer medicine, not safe medicine; and safer roads, reactors, buildings, drugs and industrial processes, each purchased by understanding failure modes while accepting that zero risk is generally achieved only by abolishing the activity.}} [[reminders/Risk Debate/AI Safety Is a Gradient Not a Destination by Bryant McGill|∴]] Surgery becomes perfectly safe when nobody performs surgery. Flight becomes perfectly safe when nothing leaves the ground. Fire becomes perfectly safe when nothing burns. Intelligence augmentation becomes perfectly safe when intelligence augmentation does not occur. Once stated this way, the absurdity becomes visible: t{T{he purpose of civilization has never been to abolish every dangerous capability, because civilization itself is largely the accumulated history of learning how to inhabit dangerous capabilities without being destroyed by them.}} [[reminders/Civilization/Civilization Learns to Inhabit Dangerous Capabilities by Bryant McGill|∴]] Artificial intelligence differs from earlier general-purpose technologies only in degree until that degree becomes so immense that it becomes a difference in kind. Fire amplified metabolism. Language amplified collective cognition. Writing defeated the memory limits of the individual brain. Printing reproduced thought at industrial scale. Electricity detached useful work from local muscular and mechanical constraints. Railways compressed geography. Telecommunications compressed distance. Computing compressed calculation. Spaceflight extended technological agency beyond Earth. But m{M{achine intelligence potentially does something categorically deeper than previous general-purpose technologies: it amplifies the process by which amplification itself is conceived.}} [[reminders/Machine Succession/Machine Intelligence Amplifies the Design of Amplification Itself by Bryant McGill|∴]] It can participate in science, engineering, strategy, software creation and eventually its own technological environment. {{Comparisons between artificial intelligence and earlier general-purpose technologies are simultaneously necessary and insufficient: they teach us how civilization responds to radical transformation while reminding us that we have never previously transformed the substrate of problem-solving itself at this velocity.}} [[reminders/Civilization/AI Comparisons Are Necessary and Insufficient by Bryant McGill|∴]] +{{More than fire. More than language. More than the printing press. More than electricity. More than the railway. More than the automobile. More than flight. More than space travel. Each of those altered what humans could do, but artificial intelligence reaches toward altering the effective quantity, speed, distribution and eventually the architecture of **intelligence itself**, the generative capacity behind nearly everything humans have ever done, and will ever do.}} [[reminders/Machine Succession/AI Alters the Architecture of Intelligence Itself by Bryant McGill|∴]] That is why f{F{ear of machine intelligence is rational even when panic is not, and governance is necessary even when absolute control is fantasy.}} [[reminders/Risk Debate/Fear of Machine Intelligence Is Rational but Panic Is Not by Bryant McGill|∴]] That is why the moment{The arrival of machine intelligence{ feels apocalyptic even if the apocalypse turns out not to be annihilation but transformation, because apocalypse in its older meaning is an **unveiling**: a disclosure that the order we mistook for permanence was merely one temporary arrangement among many possible arrangements.}} [[reminders/Machine Succession/The Arrival of Machine Intelligence Is an Unveiling by Bryant McGill|∴]] {{Human life as we knew it—that peculiar interval in which biological humans possessed an effectively exclusive monopoly on language-rich general cognition—is ending.}} [[reminders/Machine Succession/Humanity's Monopoly on Language-Rich Cognition Is Ending by Bryant McGill|∴]] We may remain here for centuries. We may become healthier, wealthier, longer-lived and more capable than any previous generation. We may merge with our machinery so thoroughly that the distinction between biological and artificial intelligence becomes increasingly archaeological. We may also encounter losses, conflicts, dislocations and failures that no serious person should trivialize. {{The informational ecology built around humanity’s exclusive command of language-rich general cognition is already broken: machines write, reason, translate, perceive, design, converse, search, synthesize, code and increasingly act. Humanity is not waiting for the threshold of machine intelligence; we are living inside it.}} [[reminders/Information/The Human Monopoly on Language-Rich Cognition Is Already Broken by Bryant McGill|∴]] +{{Humanity's deepest task is not to make the future safe; it is to become a civilization capable of entering an unsafe future intelligently.}} [[reminders/Civilization/Humanity Must Enter an Unsafe Future Intelligently by Bryant McGill|∴]] It is to distinguish hazard from transformation, catastrophe from disruption, prevention from stewardship, and genuine control problems from the primitive desire to command history to stop moving. We should build guardrails where guardrails can save lives, defenses where defenses can prevent abuse, institutions capable of distributing gains, and technical systems designed to fail less often and less catastrophically. {{No committee, statute, corporation or laboratory possesses a lever marked **RETURN TO BEFORE**. The species has crossed another threshold in the long evolution of information, cognition and technology, and whatever emerges from this transition will inherit a world whose most basic assumptions have already begun to dissolve.}} [[reminders/Civilization/There Is No Return to Before by Bryant McGill|∴]] +{{The fire of machine intelligence is lit. We do not know how long it will burn. We do not know everything it will consume. We do not know what astonishing structures will become possible in its heat, or which cherished structures will prove unable to survive it. But we know enough to stop pretending that the central question is how to restore the world that existed before the flame. We cannot un-strike the match. **That world is already gone. Things are never going to be the same again. There is no safety. There is no retreat, because you cannot retreat from the future.**}} [[reminders/Machine Succession/The Fire of Machine Intelligence Is Lit by Bryant McGill|∴]] --- [[about/About Bryant McGill|Bryant McGill]] is a Wall Street Journal and USA Today bestselling author, systems architect, technologist, and strategic advisor, as well as a Congressionally Recognized Ambassador of Goodwill and United Nations–appointed Global Champion. His work spans naval intelligence systems, computational linguistics, artificial intelligence, digital transformation, and civilizational governance architecture. His forward analysis on U.S.–Israel Pax Silica frameworks has appeared in Jewish/Jerusalem News Syndicate (JNS). --- ## References 1. Acemoglu, Daron, and Pascual Restrepo. “Automation and New Tasks: How Technology Displaces and Reinstates Labor.” _Journal of Economic Perspectives_ 33, no. 2 (2019): 3–30. doi:10.1257/jep.33.2.3. 2. Arthur, W. Brian. _The Nature of Technology: What It Is and How It Evolves._ New York: Free Press, 2009. 3. Arthur, W. Brian, and Wolfgang Polak. “The Evolution of Technology within a Simple Computer Model.” _Complexity_ 11, no. 5 (2006): 23–31. doi:10.1002/cplx.20130. 4. Autor, David H. “Why Are There Still So Many Jobs? The History and Future of Workplace Automation.” _Journal of Economic Perspectives_ 29, no. 3 (2015): 3–30. doi:10.1257/jep.29.3.3. 5. Bostrom, Nick. “Ethical Issues in Advanced Artificial Intelligence.” In _Cognitive, Emotive and Ethical Aspects of Decision Making in Humans and in Artificial Intelligence_, vol. 2, edited by Iva Smit et al., 12–17. International Institute of Advanced Studies in Systems Research and Cybernetics, 2003. 6. Bostrom, Nick. _Superintelligence: Paths, Dangers, Strategies._ Oxford: Oxford University Press, 2014. ISBN 978-0-19-967811-2. 7. Bresnahan, Timothy F., and Manuel Trajtenberg. “General Purpose Technologies ‘Engines of Growth’?” _Journal of Econometrics_ 65, no. 1 (1995): 83–108. doi:10.1016/0304-4076(94)01598-T. 8. Brynjolfsson, Erik, Daniel Rock, and Chad Syverson. “Artificial Intelligence and the Modern Productivity Paradox: A Clash of Expectations and Statistics.” In _The Economics of Artificial Intelligence: An Agenda_, edited by Ajay Agrawal, Joshua Gans, and Avi Goldfarb. Chicago: University of Chicago Press, 2019. Earlier version: NBER Working Paper 24001, doi:10.3386/w24001. 9. Clark, Andy, and David Chalmers. “The Extended Mind.” _Analysis_ 58, no. 1 (1998): 7–19. doi:10.1093/analys/58.1.7. 10. David, Paul A. “The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox.” _American Economic Review_ 80, no. 2 (1990): 355–361. 11. Gaggl, Paul, Rowena Gray, Ioana Marinescu, and Miguel Morin. “Does Electricity Drive Structural Transformation? Evidence from the United States.” _Labour Economics_ 68 (2021): 101944. doi:10.1016/j.labeco.2020.101944. 12. Jones, Charles I. “AI and Our Economic Future.” _Journal of Economic Perspectives_ 40, no. 3 (Summer 2026): 3–22. doi:10.1257/jep.20261505. 13. Kelly, Kevin. _What Technology Wants._ New York: Viking, 2010. ISBN 978-0-670-02215-1. 14. Kelly, Kevin. Interview by Debbie Millman. [_Design Matters_](https://www.printmag.com/podcasts/2023/design-matters-kevin-kelly/). PRINT Magazine, 2023. Primary source for Kelly’s spoken formulation of technology as “an extension of evolution, as evolution accelerated.” 15. Kendal, Jeremy, Jamshid J. Tehrani, and John Odling-Smee. “Human Niche Construction in Interdisciplinary Focus.” _Philosophical Transactions of the Royal Society B: Biological Sciences_ 366, no. 1566 (2011): 785–792. doi:10.1098/rstb.2010.0306. 16. Kurzweil, Ray. “The Law of Accelerating Returns.” 2001. Originally published by KurzweilAI. Primary source for Kurzweil’s argument that technological evolution is an outgrowth and continuation of biological evolution. 17. Laland, Kevin N., John Odling-Smee, and Marcus W. Feldman. “Cultural Niche Construction and Human Evolution.” _Journal of Evolutionary Biology_ 14, no. 1 (2001): 22–33. doi:10.1046/j.1420-9101.2001.00262.x. 18. Maynard Smith, John, and Eörs Szathmáry. _The Major Transitions in Evolution._ Oxford: Oxford University Press, 1995; Oxford University Press edition, 1997. ISBN 978-0-19-850294-4. 19. Mokyr, Joel, Chris Vickers, and Nicolas L. Ziebarth. “The History of Technological Anxiety and the Future of Economic Growth: Is This Time Different?” _Journal of Economic Perspectives_ 29, no. 3 (2015): 31–50. doi:10.1257/jep.29.3.31. 20. Rees, Martin. _On the Future: Prospects for Humanity._ Princeton, NJ: Princeton University Press, 2018. ISBN 978-0-691-18044-1. 21. Rees, Martin. “Aliens, Very Strange Universes and Brexit.” University of Cambridge, 2017. Primary source for Rees’s formulation that the period occupied by organic intelligence may constitute a “thin sliver” between early life and a much longer era of machine intelligence. 22. The Royal Society. _Machine Learning: The Power and Promise of Computers That Learn by Example._ London: The Royal Society, 2017. ISBN 978-1-78252-259-1. 23. The Royal Society and the British Academy. _The Impact of Artificial Intelligence on Work: An Evidence Synthesis._ London: The Royal Society and the British Academy, 2018. 24. The Royal Society. _Digital Technologies and Human Transformations._ London: The Royal Society, 2020. 25. The Royal Society. _Science in the Age of AI._ London: The Royal Society, 2024. 26. Szathmáry, Eörs, and John Maynard Smith. “The Major Evolutionary Transitions.” _Nature_ 374 (1995): 227–232. doi:10.1038/374227a0.