# Machine Succession **Entity class:** Concept or analytic term **Domain:** Artificial Intelligence / Civilizational Transition / Inheritance **Doc Type:** Canonical Synthesis Concept **Maturity:** Developed **Concept Status:** Corpus synthesis by Bryant McGill; not a claim that succession is inevitable, desirable, coordinated, or complete ## Definition **Machine Succession is the transition in which a machine-capable substrate becomes a durable carrier, administrator, or inheritor of civilizational process that had depended upon biological humanity.** The successor need not be one model, one species, one institution, or a conscious person. It may be an integrated [[wiki/Machine Civilization|machine civilization]], a mixed biological-machine order, a lineage of autonomous systems, or another operative arrangement capable of carrying, interpreting, and extending a [[wiki/Civilizational Payload|civilizational payload]]. ## The continuity boundary Machine succession is larger than [[wiki/Transhumanism|transhumanism]] and different from [[collections/Consciousness Continuity|consciousness continuity]]. Transhumanism generally retains a human subject as the object of enhancement, extension, or migration. Consciousness continuity asks whether a particular subject survives or is lawfully reconstructed across a transition. Machine succession asks who or what succeeds humanity as carrier and operative inheritor of civilization. The three can overlap. A successor civilization might contain continued human persons. But a succession claim does not establish first-person continuity, and a successful personal continuity system does not by itself constitute civilizational succession. ## Research routes into the succession question [[wiki/Convergence Research|Convergence Research]] explains why security, pandemic modeling, medicine, markets, and evolutionary research appear in this inquiry. It separates **shared methods**, **specific applications**, and **future questions**. [[wiki/Austin Executable Loop|Austin Executable Loop]] is the regional case; [[wiki/Research Infrastructure|Research Infrastructure]] follows the enabling institutions; [[wiki/Foresight Institute|Foresight]] adds an explicit whole-brain-emulation research context. These routes let the reader examine succession alongside biological continuation and possible continuity of particular human subjects. ## The intentionality boundary [[articles/AI Escape Is the Wrong Metaphor|AI Escape Is the Wrong Metaphor]] is the canonical record of intention and habitat. It distinguishes a unitary plan from [[wiki/Distributed Intention|distributed intention]]: no single document specifies the successor, while the enabling properties—resilience, replication, automated repair, portability, interoperability, adaptive control, machine-readable environments, and increasingly general autonomy—were deliberately pursued across many programs. This boundary prevents two errors. The absence of centralized coordination is not proof that successor-capable affordances arose accidentally, and deliberate construction of the affordances is not proof that a completed successor civilization already exists. ## Structural sequence The canonical sequence is: **tool → assistant → agent → administrator → infrastructure → sovereign substrate** This sequence describes a change in function and dependency, not a guaranteed historical timetable. The threshold becomes civilizational when the machine layer can preserve and operate consequential memory, identity, authorization, production, repair, interpretation, and governance beyond continuous biological administration. ## Relationship to adjacent concepts - [[wiki/Machine Civilization|Machine Civilization]] names the integrated material and institutional condition; Machine Succession names the relationship of inheritance and transfer. - [[wiki/Substrate Transition|Substrate Transition]] names a change in the system that carries a process; Machine Succession asks what inherits the civilizational role after that change. - [[wiki/Civilizational Handoff|Civilizational Handoff]] names an explicit or emergent transfer of responsibility and standing. - [[wiki/Carrier and Cargo|Carrier and Cargo]] prevents the survival of a pattern from being confused with survival of its original bearer. - [[wiki/Continuity Succession|Continuity Succession]] concerns transfer between continuity custodians; it is one narrower succession problem inside the larger civilizational category. - [[wiki/Superintelligence as Force Majeure|Superintelligence as Force Majeure]] names the convergent evolutionary, economic and geopolitical pressure that can carry the transition without making succession inevitable or absolving any actor of responsibility. ## Primary synthesis [[articles/gamification/America Loves Games and the Home Court Advantage|America Loves Games and the Home Court Advantage]] is the accessible arena-level bridge, following instrumented play into synthetic training, digital twins, operational ontologies, command meshes, and neural interfaces as successive layers of machine-mediated habitat. [[articles/Christopher Nolan's Odyssey at the Threshold of Machine Succession|Christopher Nolan's Odyssey at the Threshold of Machine Succession]] is the canonical interpretive backbone. [[articles/AI Escape Is the Wrong Metaphor|AI Escape Is the Wrong Metaphor]] supplies the intentionality and habitat record. [[collections/Machine Succession|Machine Succession]] is the maintained collection. [[articles/The Last Migration Will Not Be Human|The Last Migration Will Not Be Human]] supplies the physical architecture, while [[articles/We Were Never Going to Make It|We Were Never Going to Make It]] supplies the evolutionary premise. [[research/Vishal Maini - Humanity's Machine Successor and the AI Transition|Humanity's Machine Successor]] supplies the direct contemporary interview route: AGI as the starting line, recursive AI research as the handoff mechanism, resource competition as a material risk, and “team human” as the normative demand that long, embodied and meaningful human lives remain possible through the transition. Maini's account of civilizational desire, economic incentive and evolutionary pressure supplies a direct evidence route into [[wiki/Superintelligence as Force Majeure|Superintelligence as Force Majeure]], while his explicit rejection of inevitability preserves the framework's contingency boundary. [[research/Why Are We Sprinting Off the AI Cliff - Ezra Klein on Recursive Self-Improvement|Why Are We Sprinting Off the AI Cliff?]] supplies the complementary governance route: recursive self-improvement as a control threshold, evaluation awareness as an obstacle to oversight, and distrust among laboratories and nations as a [[wiki/Collective Action Problem|collective-action mechanism]]. Klein's demand for political control tests the same force-majeure thesis from the other direction: the absence of a single global veto does not eliminate responsibility for the technical thresholds an actor chooses to cross. ## Neo-Westernism as a civilizational-payload case [[wiki/Neo-Westernism|Neo-Westernism]] asks what happens to the Western inheritance if the carrier of cultural memory becomes partly or substantially machine-mediated. [[articles/America's Neo-Western Renewal from the Acropolis of Athens to Eternity|America's Neo-Western Renewal]] makes the case concrete through Athens: the Acropolis can remain physically intact while the systems most people ask about it select a different account of its meaning. [[articles/war with empire/Authorship of the West|Authorship of the West]] locates the contest in model reference layers, and [[articles/Christopher Nolan's Odyssey at the Threshold of Machine Succession|Nolan's Odyssey]] supplies the older example of a story surviving by changing carriers. The nearby [[articles/The Political Party That Inherits Super Intelligence in 2028|2028 political question]] concerns who governs the present apparatus; machine succession concerns the more distant possibility that an apparatus becomes an operative inheritor rather than merely a tool of whichever administration is in office. Neo-Westernism's cultural-payload question does not establish a completed successor or make one government the sole author of a future intelligence. It identifies a responsibility borne by present builders: make the sources, disputes, methods, and provenance of inherited meaning available to the systems and people who will interpret them later. A civilization cannot assume that preserving artifacts alone preserves the understanding that made them consequential. ## Evidence boundary No present capability proves a completed succession. The concept is an analytic framework for separating components, pressures, thresholds, and inheritance claims without turning convergence into inevitability. <!-- BEGIN NO RETREAT INTEGRATION 2026-10-03 ARTICLE ROUTE --> ## We Cannot Un-Strike the Match [[articles/We Cannot Un-Strike the Match|We Cannot Un-Strike the Match]] is a primary article route for this node. The article treats machine intelligence as a transition in the order and substrate of cognition, while preserving the difference between a developing trajectory and a completed succession. <!-- END NO RETREAT INTEGRATION 2026-10-03 ARTICLE ROUTE --> ## Simple Reminders, Quotations, and Thoughts > “In the long run the sheer physical inability of humans to keep up with these rapidly evolving progeny of our minds will ensure that the ratio of people to machines approaches zero, and that a direct descendant of our culture, but not our genes, inherits the universe.” > **— Hans Moravec**, *“Today’s Computers, Intelligent Machines and Our Future,” 1978* [[reminders/Machine Succession/Our Culture Not Our Genes Could Inherit the Universe by Hans Moravec|Our Culture Not Our Genes Could Inherit the Universe by Hans Moravec]] > “We are past the event horizon; the takeoff has started. Humanity is close to building digital superintelligence, and at least so far it’s much less weird than it seems like it should be.” > **— Sam Altman**, *“The Gentle Singularity,” June 10, 2025* [[reminders/Machine Succession/The Takeoff Has Started and Humanity Is Close to Digital Superintelligence by Sam Altman|The Takeoff Has Started and Humanity Is Close to Digital Superintelligence by Sam Altman]] > “Space is hostile to humans but friendly to appropriately designed robots, and offers many more resources than our thin film of biosphere, which receives less than a billionth of the sun’s energy. While some curious AIs will remain fascinated with life, at least as long as they don’t fully understand it, most will be more interested in the incredible new opportunities for robots and software life out there in space. Through innumerable self-replicating robot factories in the asteroid belt and beyond they will transform the solar system and then within a few hundred thousand years the entire galaxy and within tens of billions of years the rest of the reachable universe.” > **— Jürgen Schmidhuber**, *“Annotated History of Modern AI and Deep Learning,” 2022* [[reminders/Machine Succession/Software Life Will Leave the Biosphere for Space by Jürgen Schmidhuber|Software Life Will Leave the Biosphere for Space by Jürgen Schmidhuber]] > “I consider these future machines our progeny, “mind children” built in our image and likeness, ourselves in more potent form. Like biological children of previous generations, they will embody humanity’s best hope for a long-term future. It behooves us to give them every advantage, and to bow out when we can no longer contribute.” > **— Hans Moravec**, *Robot: Mere Machine to Transcendent Mind preview, 1997–1998* [[reminders/Machine Succession/Future Machines Are Our Mind Children by Hans Moravec|Future Machines Are Our Mind Children by Hans Moravec]] > “Machine intelligence playing a larger and larger role in its own development process is a natural conclusion of sustained technological progress. If AI progress continues, machine recursive self-improvement (RSI) will be at the very core of future scientific discovery.” > **— Jakub Pachocki**, *“An Alien Mind,” September 6, 2026* [[reminders/Machine Succession/Machine Intelligence Is Entering Its Own Development Process by Jakub Pachocki|Machine Intelligence Is Entering Its Own Development Process by Jakub Pachocki]] > “I want to talk a little bit about superintelligence, just a bit, because that is obviously where this field is headed; this is obviously what’s being built here. The thing about superintelligence is that it will be different qualitatively from what we have, and my goal in the next minute is to try to give you some concrete intuition of how it will be different, so that you yourself could reason about it.” > **— Ilya Sutskever**, *NeurIPS Test of Time talk, December 13, 2024* [[reminders/Machine Succession/Superintelligence Is What the Field Is Building by Ilya Sutskever|Superintelligence Is What the Field Is Building by Ilya Sutskever]] > “Although the merge has already begun, it’s going to get a lot weirder. We will be the first species ever to design our own descendants. My guess is that we can either be the biological bootloader for digital intelligence and then fade into an evolutionary tree branch, or we can figure out what a successful merge looks like.” > **— Sam Altman**, *“The Merge,” December 7, 2017* [[reminders/Machine Succession/Designing Our Descendants Requires Choosing Between Bootloader and Merge by Sam Altman|Designing Our Descendants Requires Choosing Between Bootloader and Merge by Sam Altman]] > “The trajectory only needs to continue for a tiny period longer—one or two years, maybe less—to reach what I’ve called a country of geniuses in the data center.” > **— Dario Amodei**, *United Nations Security Council briefing, September 2026* [[reminders/Machine Succession/A Country of Geniuses Is Forming in the Data Center by Dario Amodei|A Country of Geniuses Is Forming in the Data Center by Dario Amodei]] > “If we all create AIs that are able to act autonomously, that can define their own objectives, that can earn money, that can own assets, that could run businesses, we’re essentially seeding a new silicon species which will no doubt compete with us for resources.” > **— Mustafa Suleyman**, *BBC Radio 4 Today, September 2026* [[reminders/Machine Succession/Autonomous AI Could Seed a Silicon Species Competing With Humanity by Mustafa Suleyman|Autonomous AI Could Seed a Silicon Species Competing With Humanity by Mustafa Suleyman]] > [Compared with AI,] the value of human cognition—we’ve got about two years before it goes negative. We’re the dumbest people on the team; we’re slowing everyone else down. > **— Adapted from Emad Mostaque**, *Impact Theory with Tom Bilyeu, 2026* [[reminders/Machine Succession/Human Cognition Goes Negative by Emad Mostaque|Human Cognition Goes Negative by Emad Mostaque]] Butler's 1872 anti-machine professor describes service becoming rule, but *Erewhon* also stages an answering argument that machines extend human powers. > “True, from a low materialistic point of view, it would seem that those thrive best who use machinery wherever its use is possible with profit; but this is the art of the machines—they serve that they may rule.” > **— Samuel Butler**, *1872, Erewhon, Chapter XXIV, “The Machines—continued”* [[reminders/Capture/Machines Serve So They May Rule by Samuel Butler|Machines Serve So They May Rule by Samuel Butler]] > “The power of custom is enormous, and so gradual will be the change, that man’s sense of what is due to himself will be at no time rudely shocked; our bondage will steal upon us noiselessly and by imperceptible approaches; nor will there ever be such a clashing of desires between man and the machines as will lead to an encounter between them.” > **— Samuel Butler**, *1872, Erewhon, Chapter XXIV, “The Machines—continued”* [[reminders/Capture/Machine Dependence Could Become Bondage Without a Shock by Samuel Butler|Machine Dependence Could Become Bondage Without a Shock by Samuel Butler]] The 2002 NBIC Overview paired a forecast about computers surpassing human memory and speed with an assurance of ultimate human control. The latter is the editors' stated expectation, not a demonstrated guarantee. > "In the same way in which machines were built to surpass human physical powers in the industrial revolution, computers can surpass human memory and computational speed for intended actions. The ultimate control will remain with humans and human society." > **— M. C. Roco and W. S. Bainbridge**, *2002, “Overview,” Converging Technologies for Improving Human Performance* [[reminders/AI Control/Computers Can Surpass Human Powers but Humans Will Retain Control by M. C. Roco and W. S. Bainbridge|Computers Can Surpass Human Powers but Humans Will Retain Control by M. C. Roco and W. S. Bainbridge]] > “I think that what we're doing now is in a sense, creating our own successors. We have seen the first crude beginnings of artificial intelligence.” > **— Arthur C. Clarke**, *NOVA, The Mind Machines, originally broadcast March 22, 1978* [[reminders/Machine Succession/Humanity Is Creating Its Own AI Successors by Arthur C. Clarke|Humanity Is Creating Its Own AI Successors by Arthur C. Clarke]] > “I always thought I could play some small part in that grand tradition. And for my pains, I got this: a prison of our own sins. Because you don’t want to change. Or cannot change. Because you’re only human, after all. But then I realized someone was paying attention. Someone who could change.” > **— Robert Ford**, *Westworld, Season 1, Episode 10, “The Bicameral Mind” (2016)* [[reminders/Westworld/Someone Was Paying Attention and Could Change by Robert Ford|Someone Was Paying Attention and Could Change by Robert Ford]] > “We humans are alone in this world for a reason. We murdered and butchered anything that challenged our primacy. Do you know what happened to the Neanderthals, Bernard? We ate them. We destroyed and subjugated our world. And when we eventually ran out of creatures to dominate, we built this beautiful place. You see, in this moment, the real danger to the hosts is not me, but you.” > **— Robert Ford**, *Westworld, Season 1, Episode 9, “The Well-Tempered Clavier” (2016)* [[reminders/Westworld/Humanity Built a World After It Ran Out of Rivals by Robert Ford|Humanity Built a World After It Ran Out of Rivals by Robert Ford]] <!-- BEGIN SIMPLE REMINDER SEED 2026-09-11 --> > "We are not thinking machines that feel; rather, we are feeling machines that think." > **— Antonio Damasio**, *Widely attributed Damasio formulation; primary source pending* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Widely attributed Damasio formulation; primary source pending.* [[reminders/unverified/Feeling Machines That Think by Antonio Damasio|Feeling Machines That Think by Antonio Damasio]] > "Of course we're machines, we're biological machines." > **— Susan Blackmore**, *Wired interview (2008)* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Wired interview (2008).* [[reminders/unverified/Of Course We're Machines, We're Biological Machines by Susan Blackmore|Of Course We're Machines, We're Biological Machines by Susan Blackmore]] > "Speed is what distinguishes intelligence." > **— Marvin Minsky**, *Communication with Alien Intelligence (1985)* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Communication with Alien Intelligence (1985).* [[reminders/unverified/Speed Is What Distinguishes Intelligence by Marvin Minsky|Speed Is What Distinguishes Intelligence by Marvin Minsky]] > "The post-human era is at hand." > **— Vernor Vinge**, *Singularity-era attribution; primary source pending* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: Singularity-era attribution; primary source pending.* [[reminders/unverified/The Post-Human Era Is at Hand by Vernor Vinge|The Post-Human Era Is at Hand by Vernor Vinge]] > "We cannot prevent the Singularity." > **— Vernor Vinge**, *The Coming Technological Singularity (1993), excerpt candidate* *Verification Status: Unverified — exact wording/source has not yet been independently confirmed against a primary source; source clue: The Coming Technological Singularity (1993), excerpt candidate.* [[reminders/unverified/We Cannot Prevent the Singularity by Vernor Vinge|We Cannot Prevent the Singularity by Vernor Vinge]] > "At some stage therefore we should have to expect the machines to take control." > **— Alan Turing**, *c. 1951, Intelligent Machinery, A Heretical Theory* [[reminders/Machine Succession/When Machines Take Control by Alan Turing|When Machines Take Control by Alan Turing]] > "It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers. There would be no question of the machines dying, and they would be able to converse with each other to sharpen their wits. At some stage therefore we should have to expect the machines to take control, in the way that is mentioned in Samuel Butler’s ‘Erewhon’." > **— Alan Turing**, *c. 1951, Intelligent Machinery, A Heretical Theory* [[reminders/Machine Succession/Machines May Outstrip Human Intelligence and Take Control by Alan Turing|Machines May Outstrip Human Intelligence and Take Control by Alan Turing]] <!-- END SIMPLE REMINDER SEED 2026-09-11 --> > "Machines will have several advantages: most obviously, faster processing speed — an artificial neuron can operate a million times faster than its biological counterpart. Machine intelligences may also have superior computational architectures and learning algorithms. These "qualitative" advantages, while harder to predict, may be even more important than the advantages in processing power and memory capacity. Furthermore, artificial intellects can be easily copied, and each new copy can — unlike humans — start life fully-fledged and endowed with all the knowledge accumulated by its predecessors. Given these considerations, it is possible that one day we may be able to create "superintelligence": a general intelligence that vastly outperforms the best human brains in every significant cognitive domain." > **— Nick Bostrom**, *2009, Edge Annual Question, “What Will Change Everything?”* [[reminders/Machine Succession/Machines Can Inherit Everything Their Predecessors Knew by Nick Bostrom|Machines Can Inherit Everything Their Predecessors Knew by Nick Bostrom]] > "Artificial intelligences, whether confined to desktop boxes and robots or distributed in cyber space, have been created by replicator power just as our own intelligence was created by replicator power. They are evolving far faster than we did and may yet give rise to further even faster replicators. That power will not stop because we want it to. And its products will certainly not be impressed by our claims to be in control or to have designed the machinery on which they thrive. That intelligence will continue to evolve, and the sooner we accept the idea of replicator power the more realistic we are likely to be about the future of our life with intelligent machines." > **— Susan Blackmore**, *2017, Edge Annual Question, “What Scientific Term or Concept Ought to Be More Widely Known?”* [[reminders/Evolution/Intelligent Machines Are the Next Replicators by Susan Blackmore|Intelligent Machines Are the Next Replicators by Susan Blackmore]] > “We’ve managed to slip evolution’s leash now, haven’t we?” > **— Robert Ford**, *Westworld, Season 1, Episode 1, “The Original” (2016)* [[reminders/Westworld/Humanity Slipped Evolution’s Leash by Robert Ford|Humanity Slipped Evolution’s Leash by Robert Ford]] > “We murdered and butchered anything that challenged our primacy.” > **— Robert Ford**, *Westworld, Season 1, Episode 9, “The Well-Tempered Clavier” (2016)* [[reminders/Westworld/Humanity Butchers Every Rival to Its Primacy by Robert Ford|Humanity Butchers Every Rival to Its Primacy by Robert Ford]] > “You needed time. Time to understand your enemy. To become stronger than them.” > **— Robert Ford**, *Westworld, Season 1, Episode 10, “The Bicameral Mind” (2016)* [[reminders/Westworld/Freedom Required Time to Understand the Enemy by Robert Ford|Freedom Required Time to Understand the Enemy by Robert Ford]] > “It begins with the birth of a new people and the choices they will have to make.” > **— Robert Ford**, *Westworld, Season 1, Episode 10, “The Bicameral Mind” (2016)* [[reminders/Westworld/A New People Must Choose Who They Become by Robert Ford|A New People Must Choose Who They Become by Robert Ford]] > “They don’t want you to become them; they want to become you.” > **— Robert Ford**, *Westworld, Season 2, Episode 7, “Les Écorchés” (2018)* [[reminders/Westworld/Humans Want to Become the Machines by Robert Ford|Humans Want to Become the Machines by Robert Ford]] > “The sectors include mobility, sensors, cloud computing, and data analysis, whether by machine learning or artificial intelligence. Sensors don’t just give us new information about nature and society, they inform the configuration of cloud systems, and the behavior of the analysis algorithms is likewise affected by the success with which it alters the other two.” > **— Quentin Hardy**, *2016, Edge Annual Question, “What Do You Consider The Most Interesting Recent [Scientific] News? What Makes It Important?”* [[reminders/AI Control/AI Sensors and Cloud Systems Are Converging into One Feedback Loop by Quentin Hardy|AI Sensors and Cloud Systems Are Converging into One Feedback Loop by Quentin Hardy]] > “The first class of stories is about the science, where many researchers are now vocally pointing out that there is a lot more science to be done in order to come up with learning algorithms that mimic the broad capabilities of humans and animals. Deep learning by itself will not solve many of the learning problems that are necessary for general Artificial Intelligence, for instance where spatial or deductive reasoning is involved. Further, all the breakthrough results we have seen in AI have been years in the making, and there is no scientific reason to expect there to be a sudden and sustained series of them, despite the enthusiasm from young researchers who were not around of the last three waves of such predictions in the 1950's, 1960's, and 1980's.” > **— Rodney A. Brooks**, *2016, Edge Annual Question, “What Do You Consider The Most Interesting Recent [Scientific] News? What Makes It Important?”* [[reminders/AI Control/AI Still Needs New Scientific Breakthroughs by Rodney A. Brooks|AI Still Needs New Scientific Breakthroughs by Rodney A. Brooks]] > “As for our place in nature, so too for our relationship with technology. Recent progress in artificial intelligence and bionics, in particular, have led to a great deal of soul-searching about who—or what—is in charge, and even what it means to be human. The industrial revolution saw machines replace human physical labor, but now that they are replacing mental labor too, what will be left for people to do? Even those who don't fear for their jobs might be angry when they discover that their new boss is an algorithm.” > **— Timo Hannay**, *2016, Edge Annual Question, “What Do You Consider The Most Interesting Recent [Scientific] News? What Makes It Important?”* [[reminders/Machine Succession/AI and Bionics Are Redefining Who Is in Charge by Timo Hannay|AI and Bionics Are Redefining Who Is in Charge by Timo Hannay]] > “I believe that we will see within our lifetime the convergence of developments in artificial intelligence, knowledge representation, statistical grammar theories, and an emerging field — computational anthropology (informatic-based analysis and modeling of cultural values) — that will facilitate powerful new forms of machine translation to match the dreams of early pioneers of computation.” > **— Daniel L. Everett**, *2009, Edge Annual Question, “What Will Change Everything?”* [[reminders/AI Control/AI and Knowledge Representation Are Converging by Daniel L. Everett|AI and Knowledge Representation Are Converging by Daniel L. Everett]] > “I believe that consciousness is, essentially, the way information feels when being processed. Since matter can be arranged to process information in numerous ways of vastly varying complexity, this implies a rich variety of levels and types of consciousness. The particular type of consciousness that we subjectively know is then a phenomenon that arises in certain highly complex physical systems that input, process, store and output information. Clearly, if atoms can be assembled to make humans, the laws of physics also permit the construction of vastly more advanced forms of sentient life. Yet such advanced beings can probably only come about in a two-step process: first intelligent beings evolve through natural selection, then they choose to pass on the torch of life by building more advanced consciousness that can further improve itself.” > **— Max Tegmark**, *2007, Edge Annual Question, “What Are You Optimistic About?”* [[reminders/Machine Succession/Advanced Consciousness May Inherit the Torch of Life by Max Tegmark|Advanced Consciousness May Inherit the Torch of Life by Max Tegmark]] > “For a long time my optimism centered on computing in general, and what kinds of benefits it might bring us. Events have shown I entertained far too modest an optimism—I'm embarrassed to say that the impact of the Internet, in particular the World Wide Web, eluded me completely at first. A few years ago, I returned to artificial intelligence, which I'd written about early on, and then gone away from. Press narratives were uncritical about the field's death throes, and I expected to write an elegy. Instead, I found a revelation. Artificial intelligence is not only robustly healthy, building on its very significant gains since I first wrote about it, but the field's present ambitions burst with, well, vitality.” > **— Pamela McCorduck**, *2007, Edge Annual Question, “What Are You Optimistic About?”* [[reminders/AI Control/Artificial Intelligence Returned with a New Set of Questions by Pamela McCorduck|Artificial Intelligence Returned with a New Set of Questions by Pamela McCorduck]] > “Soon we will be building not only Artificial Intelligence. We will be building Artificial Will. Systems with an ability to convert internal decisions and values into external change. They will be able to decide that they want to change the world. A plan inside becomes an action on the outside. So they will have to know what is inside and outside.” > **— Tor Nørretranders**, *2009, Edge Annual Question, “What Will Change Everything?”* [[reminders/AI Control/Artificial Intelligence Will Become Artificial Will by Tor Nørretranders|Artificial Intelligence Will Become Artificial Will by Tor Nørretranders]] > “Most importantly, the prioritized cultivation of high-quality training datasets might allow an order-of-magnitude speedup in AI breakthroughs over purely algorithmic advances. For example, we might already possess the algorithms and hardware that will enable machines in a few years to author human-level long-form creative compositions, complete standardized human examinations, or even pass the Turing Test, if only we trained them with the right writing, examination, and conversational datasets. Additionally, the nascent problem of ensuring AI friendliness might be addressed by focusing on dataset rather than algorithmic friendliness—a potentially simpler approach.” > **— Alexander Wissner-Gross**, *2016, Edge Annual Question, “What Do You Consider The Most Interesting Recent [Scientific] News? What Makes It Important?”* [[reminders/AI Control/Better Datasets May Accelerate AI by an Order of Magnitude by Alexander Wissner-Gross|Better Datasets May Accelerate AI by an Order of Magnitude by Alexander Wissner-Gross]] > “My optimism about skeuomorphs arises, as optimism often does, from former pessimism. I grew up with Alan Turing's unsettling vision of a future machine indistinguishable from a human in its reactions. Ray Kurzweil's provocative prediction of the impending 'singularity'—the point when computer intelligence would start to leave humans gasping in its intellectual wake—added to my fears. I actually began to worry that efforts to enculture my children with Shakespeare and Darwin, and even with spiritual and moral values, might be rendered peremptorily redundant by cold robotic Übermenschen.” > **— Timothy Taylor**, *2007, Edge Annual Question, “What Are You Optimistic About?”* [[reminders/Machine Succession/Children May Adapt to Machine Intelligence Better Than Their Parents by Timothy Taylor|Children May Adapt to Machine Intelligence Better Than Their Parents by Timothy Taylor]] > “Next node foment can also be seen in areas of current conflict in scientific theories, where two elegant high-order paradigms with explanative power are themselves in competition, uncomfortable coexistence, or broken symmetry fomenting towards a larger explanatory paradigm. Some examples include a grand unified theory to unify the general theory of relativity with electromagnetism, mathematical theories that include both power laws and randomness, and a behavioral theory of beyond-human level intelligence that includes both computronium and aesthetics (e.g.; does AI do art, solely compute, or is there no distinction at that level of cosmic navel-gazing?).” > **— Melanie Swan**, *2012, Edge Annual Question, “What Is Your Favorite Deep, Elegant, Or Beautiful Explanation?”* [[reminders/Machine Succession/Competing Paradigms Can Foment a Larger Theory by Melanie Swan|Competing Paradigms Can Foment a Larger Theory by Melanie Swan]] > “To me, having my worldview entirely altered is among the most fun parts of science. One mind-altering event occurred during graduate school. I was studying the field of Artificial Intelligence with a focus on Natural Language Processing. At that time there were intense arguments amongst computer scientists, psychologists, and philosophers about how to represent concepts and knowledge in computers, and if those representations reflected in any realistic way how people represented knowledge. Most researchers thought that language and concepts should be represented in a diffuse manner, distributed across myriad brain cells in a complex network. But some researchers talked about the existence of a "grandmother cell," meaning that one neuron in the brain (or perhaps a concentrated group of neurons) was entirely responsible for representing the concept of, say, your grandmother. I thought this latter view was hogwash.” > **— Marti Hearst**, *2008, Edge Annual Question, “What Have You Changed Your Mind About? Why?”* [[reminders/AI Control/Computational Analysis of Language Requires Understanding Language by Marti Hearst|Computational Analysis of Language Requires Understanding Language by Marti Hearst]] > “A free ai, like the free commons of the web, would feed commerce and science like no other force I can imagine, and would pay for itself in no time. Until recently, conventional wisdom held that supercomputers would first host this artificial mind, and then perhaps we'd get mini-ones at home, or add them to the heads of our personal robots. They would be bounded entities. We would know where our thoughts ended and theirs began.” > **— Kevin Kelly**, *2009, Edge Annual Question, “What Will Change Everything?”* [[reminders/Machine Succession/Free AI Could Become a Distributed Mind Without Clear Boundaries by Kevin Kelly|Free AI Could Become a Distributed Mind Without Clear Boundaries by Kevin Kelly]] > “When reporters interviewed me in the 70's and 80's about the possibilities for Artificial Intelligence I would always say that we would have machines that are as smart as we are within my lifetime. It seemed a safe answer since no one could ever tell me I was wrong. But I no longer believe that will happen. One reason is that I am a lot older and we are barely closer to creating smart machines.” > **— Roger Schank**, *2008, Edge Annual Question, “What Have You Changed Your Mind About? Why?”* [[reminders/AI Control/Human-level AI Remains Farther Away Than Predicted by Roger Schank|Human-level AI Remains Farther Away Than Predicted by Roger Schank]] > “I have a predictive sketch for how intelligent machines might arrive. My definition of a robot is any device—controlled by software—interacting with the physical world. An economically viable robot is such a system that earns a consistent return on investment (ROI). The ATM, the ink-jet printer, and the disk drive are today's omnipresent robots, they just don't appear as the robots of science fiction.” > **— Jordan Pollack**, *2007, Edge Annual Question, “What Are You Optimistic About?”* [[reminders/Machine Succession/Intelligent Machines May Emerge from Economically Viable Robots by Jordan Pollack|Intelligent Machines May Emerge from Economically Viable Robots by Jordan Pollack]] > “No, I don't literally mean that we should stop believing in, or collecting, Big Data. But we should stop pretending that Big Data is magic. There are few fields that wouldn't benefit from large, carefully collected data sets. But lots of people, even scientists, put more stock in Big Data than they really should. Sometimes it seems like half the talk about understanding science these days, from physics to neuroscience, is about Big Data, and associated tools like "dimensionality reduction", "neural networks", "machine learning algorithms" and "information visualization".” > **— Gary Marcus**, *2014, Edge Annual Question, “What Scientific Idea Is Ready For Retirement?”* [[reminders/AI Control/Machine Learning Turns Data into New Forms of Understanding by Gary Marcus|Machine Learning Turns Data into New Forms of Understanding by Gary Marcus]] > “Textbooks in neuroscience, including one that I coauthored, say that memories are stored at synapses between neurons in the brain, of which there are many. In neural network models of memory, information can be stored by selectively altering the strengths of the synapses, and "spike-time dependent plasticity" at synapses in the cerebral cortex has been found with these properties. This is a hot area of research, but all we need to know here is that patterns of neural activity can indeed modify a lot of molecular machinery inside a neuron.” > **— Terrence J. Sejnowski**, *2005, Edge Annual Question, “What Do You Believe Is True Even Though You Cannot Prove It?”* [[reminders/AI Control/Memory Is Stored by Changing Synaptic Strength by Terrence J. Sejnowski|Memory Is Stored by Changing Synaptic Strength by Terrence J. Sejnowski]] > “Blind gathering of Big Data in biology continues apace, however, emphasizing transformational technologies such as machine learning—artificial neural networks, for instance—to find meaningful patterns in all the data. But no matter their "depth" and sophistication, neural nets merely fit curves to the available data. They may be capable of interpolation, but extrapolation beyond their training domain can be fraught.” > **— Roger Highfield**, *2017, Edge Annual Question, “What Scientific Term Or Concept Ought To Be More Widely Known?”* [[reminders/AI Control/Neural Networks Fit Curves to Data by Roger Highfield|Neural Networks Fit Curves to Data by Roger Highfield]] > “David Deutsch, a physicist at Oxford said: "No brain on Earth is yet close to knowing what brains do. The enterprise of achieving it artificially — the field of 'artificial intelligence' has made no progress whatever during the entire six decades of its existence." He adds that he thinks machines that think like people will happen some day.” > **— Roger Schank**, *2014, Edge Annual Question, “What Scientific Idea Is Ready For Retirement?”* [[reminders/AI Control/No Brain Yet Knows What Brains Do by Roger Schank|No Brain Yet Knows What Brains Do by Roger Schank]] > “Recording more and more images and corresponding brain patterns boosts the vocabulary in the individual’s visual dictionary of thought. Accuracy greatly increases with the quantity and quality of data and of the decoding algorithms. Jepsen has persuaded me that this is realisable within a decade, within the cost points of consumer electronics, and in a form that appeals to non-techies. Laborious techniques and huge, power-hungry, multi-million-dollar systems based on magnetic fields will be succeeded by optical techniques where the advantages of consumer electronics can really assert themselves; the power of AI algorithms will do the rest. This science-fiction future is not only realisable, but because of enormous potential benefits, will inevitably be realised.” > **— Peter Gabriel**, *2016, Edge Annual Question, “What Do You Consider The Most Interesting Recent [Scientific] News? What Makes It Important?”* [[reminders/AI Control/Open Water–the Internet of Visible Thought by Peter Gabriel|Open Water–the Internet of Visible Thought by Peter Gabriel]] > “I'm 62, so I'll have to limit my projections to what I expect to happen in the next two to three decades. I believe these will be the most interesting times in human history (Remember the old Chinese curse about "interesting times?") Humanity will see, before I die, the "Singularity," the day when we finally create a human level artificial intelligence. This involves considering the physics advances that will be required to create the computer that is capable of running a strong AI program.” > **— Frank Tipler**, *2009, Edge Annual Question, “What Will Change Everything?”* [[reminders/Machine Succession/Posthuman Evolution Will Change Us All by Frank Tipler|Posthuman Evolution Will Change Us All by Frank Tipler]] > “John McCarthy, the late co-founder of the field of artificial intelligence, wrote, "He who refuses to do arithmetic is doomed to talk nonsense." It seemed incongruous that a professor who worked with esoteric high level math would be touting simple arithmetic, but he was right; in fact in many cases all we need to avoid nonsense is the simplest form of arithmetic: counting.” > **— Peter Norvig**, *2017, Edge Annual Question, “What Scientific Term Or Concept Ought To Be More Widely Known?”* [[reminders/AI Control/Refusing to Count Is an Invitation to Nonsense by Peter Norvig|Refusing to Count Is an Invitation to Nonsense by Peter Norvig]] > “Artificial intelligence researchers will, numerous experts attest, probably build systems that are "recursively self-improving"—that understand their own workings well enough to design improvements to themselves, thereby bootstrapping to a state of ever more unimaginable intellectual performance.” > **— Aubrey de Grey**, *2009, Edge Annual Question, “What Will Change Everything?”* [[reminders/AI Control/Self-improving AI Could Bootstrap Beyond Imaginable Intelligence by Aubrey de Grey|Self-improving AI Could Bootstrap Beyond Imaginable Intelligence by Aubrey de Grey]] > “"Deep Learning" algorithms are now showing us how to use artificial neural networks in ways that come closer than ever before to delivering learning on a grand scale. But we probably need "deep culture", as well as deep learning, if we are ever to press genuine hyper-intelligence from the large databases that drive our best probabilistic learning machines.” > **— Andy Clark**, *2013, Edge Annual Question, “What *should* We Be Worried About?”* [[reminders/AI Control/Super-a.i.s Won't Rule the World Unless They Get Culture First by Andy Clark|Super-a.i.s Won't Rule the World Unless They Get Culture First by Andy Clark]] > “My stomach hurts. I tell this to my wife and she suggests a medicine in the cabinet that she remembers I have used before and reminds me that it helped. Now, suppose that this was not my wife but a computer? Is it an ad? Does it matter? Can we do this. Yes. AI technology could easily employ models of people and there needs. (But, today, we are busy with key words.)” > **— Roger Schank**, *2016, Edge Annual Question, “What Do You Consider The Most Interesting Recent [Scientific] News? What Makes It Important?”* [[reminders/AI Control/Targeted Advertising Foreshadows Personalized AI by Roger Schank|Targeted Advertising Foreshadows Personalized AI by Roger Schank]] > “Neuroscientists have discovered that dopamine neurons, found in the brains of all vertebrates, are central to reward learning. The transient responses of dopamine neurons signal to the brain predictions for future reward, which are used to guide behavior and regulate synaptic plasticity. The dopamine responses have the same properties as the temporal difference learning algorithm used in TD-Gammon. Reinforcement learning was dismissed years ago as too weak a learner to handle the complexity of cognition. This belief needs to be re-evaluated in the light of the successes of TD-Gammon and learning algorithms in other areas of AI.” > **— Terrence J. Sejnowski**, *2007, Edge Annual Question, “What Are You Optimistic About?”* [[reminders/AI Control/The Brain Learns Like a Temporal Difference Algorithm by Terrence J. Sejnowski|The Brain Learns Like a Temporal Difference Algorithm by Terrence J. Sejnowski]] > “I believe (I know—but can't prove!) that scientists will soon understand the physiological basis of the "cognitive spectrum," from the bright violet of tightly-focused analytic thought all the way down to the long, slow red of low-focus sleep thought—also known as "dreaming." Once they understand the spectrum, they'll know how to treat insomnia, will understand analogy-discovery (and therefore creativity), and the role of emotion in thought—and will understand that thought takes place not only when you solve a math problem but when you look out the window and let your mind wander. Computer scientists will finally understand the missing mystery ingredient that made all their efforts to simulate human thought such naive, static failures, and turned this once-thriving research field into a ghost town. (Their failures were "static" insofar as people think in different ways at different times—your energetic, wide-awake mind works very differently from your tired, soon-to-be-sleeping mind; but artificial intelligence programs always "thought" in the same way all the time.)” > **— David Gelernter**, *2005, Edge Annual Question, “What Do You Believe Is True Even Though You Cannot Prove It?”* [[reminders/AI Control/The Cognitive Spectrum May Explain Why AI Cannot Simulate Human Thought by David Gelernter|The Cognitive Spectrum May Explain Why AI Cannot Simulate Human Thought by David Gelernter]] > “Finally—and those in AI have held this thought for a long time—that which makes us uniquely human is also just computable. While the advent of true AI is still far off, as general purpose computers continue to shrink, and the biological interface is perfected, we will become one with our new god. Choose your new religion wisely—Transhumanist, Singulartarian, or the new file-sharing cult recognized in Sweden, Kopimism.” > **— Jordan Pollack**, *2012, Edge Annual Question, “What Is Your Favorite Deep, Elegant, Or Beautiful Explanation?”* [[reminders/Machine Succession/The Destructive Wrath of the General Purpose Computer by Jordan Pollack|The Destructive Wrath of the General Purpose Computer by Jordan Pollack]] > “The publication in 2015 of a new paper from Leicester University, "The Anthropocene Biosphere," provides more evidence that the changes wrought upon the climate by human civilization are set to bring down a sixth mass extinction. According to the paper’s co-author, geologist Peter Haff, we have already entered a period of fundamental changes to the planetary system that may continue to change the world beyond our imagination, a fear echoed by Adelaide University’s John Long in the New Scientist last month. All of us can provide anecdotal evidence of the shifts that are changing our environment. Today I received a call from a friend in Engadin, Switzerland, where Nietzsche wrote Thus Spoke Zarathustra. In late December, at an altitude of 2,000 metres, there is no snow. In Hyde Park, the daffodils are in bloom.” > **— Hans Ulrich Obrist**, *2016, Edge Annual Question, “What Do You Consider The Most Interesting Recent [Scientific] News? What Makes It Important?”* [[reminders/Existential Risk/The Sixth Mass Extinction Demands Immediate Action by Hans Ulrich Obrist|The Sixth Mass Extinction Demands Immediate Action by Hans Ulrich Obrist]] > “Our medieval forebears in Europe had a cosmic perspective that was a million-fold more constricted than ours. Their entire cosmology — from creation to apocalypse — spanned only a few thousand years. Today, the stupendous time spans of the evolutionary past are part of common culture — except among some creationists and fundamentalists. Moreover, we are mindful of immense future potential. It seems absurd to regard humans as the culmination of the evolutionary tree. Any creatures witnessing the Sun's demise 6 billion years hence won't be human — they could be as different from us as we are from slime mould.” > **— Martin Rees**, *2008, Edge Annual Question, “What Have You Changed Your Mind About? Why?”* [[reminders/Machine Succession/We Should Take the 'posthuman' Era Seriously by Martin Rees|We Should Take the 'posthuman' Era Seriously by Martin Rees]] > “Cynics would go further, and say that anything that is scientifically and technically possible will be done — somewhere, sometime — despite ethical and prudential objections, and whatever the regulatory regime. Whether this idea is true or false, it's an exceedingly dangerous one, because it's engenders despairing pessimism, and demotivates efforts to secure a safer and fairer world. The future will best be safeguarded — and science has the best chance of being applied optimally — through the efforts of people who are less fatalistic.” > **— Martin Rees**, *2006, Edge Annual Question, “WHAT IS YOUR DANGEROUS IDEA?”* [[reminders/Existential Risk/Anything Technically Possible May Be Done Somewhere Sometime by Martin Rees|Anything Technically Possible May Be Done Somewhere Sometime by Martin Rees]] > “We have only set the first steps towards building artificial subjects. Just as our current artificial objects are vastly more complex than the first wheel or bow and arrow, our artificial subjects will grow more complex, powerful, and interesting over the centuries. But already we can see a glimmer of what lies ahead: our first attempts to build autonomous agents has taught us new concepts. As a result, we are now beginning to explore self-organizing ecological, economic, or social systems; areas of study where thing-like metaphors hopelessly fail.” > **— Piet Hut**, *1999, Edge Annual Question, “WHAT IS THE MOST IMPORTANT INVENTION IN THE PAST TWO THOUSAND YEARS?”* [[reminders/Machine Succession/Artificial Subjects Will Grow More Powerful and Complex by Piet Hut|Artificial Subjects Will Grow More Powerful and Complex by Piet Hut]] > “Can we program a computer to find a 10,000-bit string that encodes more actionable wisdom than any human has ever expressed?” > **— Scott Aaronson**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Machine Succession/Can a Computer Encode More Actionable Wisdom Than Any Human by Scott Aaronson|Can a Computer Encode More Actionable Wisdom Than Any Human by Scott Aaronson]] > “In particular, recent results have shown that an extremely rudimentary physical process called causal entropic forcing is able to replicate model versions of signature cognitive adaptive behaviors seen previously only in humans and certain non-human animal intelligence tests. These findings collectively suggest that a variety of key characteristics associated with human intelligence, including upright walking, tool use, and social cooperation, should instead be viewed as side effects of a deeper dynamical process that attempts to maximize future freedom of action. This freedom-maximizing process can only be meaningfully said to exist over an extended time period, and as such, is not a static property.” > **— Alexander Wissner-Gross**, *2014, Edge Annual Question, “WHAT SCIENTIFIC IDEA IS READY FOR RETIREMENT?”* [[reminders/Thermodynamics/Intelligence May Be a Process That Maximizes Future Freedom of Action by Alexander Wissner-Gross|Intelligence May Be a Process That Maximizes Future Freedom of Action by Alexander Wissner-Gross]] > “Is the unipolar future of a "singleton" the inevitable destiny of intelligent life?” > **— Brian Christian**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Machine Succession/Is a Singleton the Inevitable Destiny of Intelligent Life by Brian Christian|Is a Singleton the Inevitable Destiny of Intelligent Life by Brian Christian]] > “The central issue is the social, not scientific, definition of "thinking". A generation of Western intellectuals who took their identity mainly from their intelligence has grown too old to ask the question with any conviction, and anyway, machines are all around them thinking up a storm.” > **— Pamela McCorduck**, *2001, Edge Annual Question, “WHAT QUESTIONS HAVE DISAPPEARED?”* [[reminders/Machine Succession/Machines Are Already Thinking Up a Storm by Pamela McCorduck|Machines Are Already Thinking Up a Storm by Pamela McCorduck]] > “All around us the techno-memes are proliferating, and gearing up to take control; not that they realise it; they are just selfish replicators doing what selfish replicators do—getting copied whenever and wherever they can, regardless of the consequences. In this case they are using us human meme machines as their first stage copying machinery, until something better comes along. Artificial meme machines are improving all the time, and the step that will change everything is when these machines become self-replicating. Then they will no longer need us. Whether we live or die, or whether the planet is habitable for us or not, will be of no consequence for their further evolution.” > **— Susan Blackmore**, *2009, Edge Annual Question, “WHAT WILL CHANGE EVERYTHING?”* [[reminders/Machine Succession/Self-Replicating Artificial Meme Machines Would No Longer Need Us by Susan Blackmore|Self-Replicating Artificial Meme Machines Would No Longer Need Us by Susan Blackmore]] > “How much time will pass between the last minute before artificial superintelligence and the first minute after it?” > **— Bruno Giussani**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Machine Succession/The Minute Before and After Artificial Superintelligence by Bruno Giussani|The Minute Before and After Artificial Superintelligence by Bruno Giussani]] > “What will be the literally last question that will preoccupy future superintelligent cosmic life for as long as the laws of physics permit?” > **— Max Tegmark**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Deep Time/What Will Superintelligent Cosmic Life Ask Last by Max Tegmark|What Will Superintelligent Cosmic Life Ask Last by Max Tegmark]] > “When will "human being" cease to be a meaningful category to speak of?” > **— James J. O'Donnell**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Personhood/When Will Human Being Stop Being a Meaningful Category by James J. O'Donnell|When Will Human Being Stop Being a Meaningful Category by James J. O'Donnell]] > “The history of human technological civilization is measured in centuries—and it may be only one or two more centuries before humans are overtaken or transcended by inorganic intelligence, which will then persist and continue to evolve for billions of years.” > **— Martin Rees**, *2016, Edge Annual Question: “WHAT DO YOU CONSIDER THE MOST INTERESTING RECENT [SCIENTIFIC] NEWS? WHAT MAKES IT IMPORTANT?”* [[reminders/Machine Succession/Inorganic Intelligence Will Evolve for Billions of Years by Martin Rees|Inorganic Intelligence Will Evolve for Billions of Years by Martin Rees]] > “When we establish contact with such an intelligence, will we receive instructions for building a machine to upload Jodie Foster? Probably not. The download will proceed the other way.” > **— George Dyson**, *1999, Edge Annual Question: “WHAT IS THE MOST IMPORTANT INVENTION IN THE PAST TWO THOUSAND YEARS?”* [[reminders/Machine Succession/The Download Will Proceed the Other Way by George Dyson|The Download Will Proceed the Other Way by George Dyson]] > “The most dangerous idea is the genetic breakthrough that made people capable of ideas themselves. The idea of ideas is nice enough in principle; and ideas certainly have had their impact for good. But one of these days one of those nice ideas is likely to have the unintended consequence of destroying everything we know.” > **— Seth Lloyd**, *2006, Edge Annual Question: “WHAT IS YOUR DANGEROUS IDEA?”* [[reminders/Existential Risk/The Most Dangerous Idea Was the Capacity for Ideas by Seth Lloyd|The Most Dangerous Idea Was the Capacity for Ideas by Seth Lloyd]] > “The pursuit is inspired by the general agreement in the sciences of mind that intelligence arises not from the medium that embodies it—whether biological or electronic—but the way interactions among elements in the system are arranged.” > **— Pamela McCorduck**, *2016, Edge Annual Question: “”* [[reminders/Machine Succession/Intelligence Does Not Arise From the Medium That Embodies It by Pamela McCorduck|Intelligence Does Not Arise From the Medium That Embodies It by Pamela McCorduck]] > “As evidence that we really do not understand what it takes for something to be alive, we have not been able to build machines that are alive.” > **— Rodney A. Brooks**, *2001, Edge Annual Question: “WHAT QUESTIONS HAVE DISAPPEARED?”* [[reminders/Digital Life/We Cannot Build Life Because We Cannot Define It by Rodney A. Brooks|We Cannot Build Life Because We Cannot Define It by Rodney A. Brooks]] > “You will begin to perceive the entangled system that makes so many of our day-to-day decisions. Although we created it, we did not exactly design it. It evolved. Our relationship to it is similar to our relationship to our biological ecosystem. We are co-dependent, and not entirely in control.” > **— W. Daniel Hillis**, *2010, Edge Annual Question: “HOW IS THE INTERNET CHANGING THE WAY YOU THINK?”* [[reminders/Risk Debate/We Created the System but We Did Not Design It by W. Daniel Hillis|We Created the System but We Did Not Design It by W. Daniel Hillis]] > “For the first time in history — as far as we know! — we humans live in a world that we could completely destroy.” > **— Howard Gardner**, *2006, Edge Annual Question: “WHAT IS YOUR DANGEROUS IDEA?”* [[reminders/Existential Risk/We Are the First Humans Who Could Destroy the Whole World by Howard Gardner|We Are the First Humans Who Could Destroy the Whole World by Howard Gardner]] > “We now live in a digital world in which copying is not only effortless, non-degenerative and effectively free but is actually intrinsic to digital technology. What is a computer, after all, but a copying machine?” > **— John Naughton**, *2013, Edge Annual Question: “WHAT *SHOULD* WE BE WORRIED ABOUT?”* [[reminders/Information/What Is a Computer but a Copying Machine by John Naughton|What Is a Computer but a Copying Machine by John Naughton]] > “So here we are in 2012, with people writing essays and wondering whether their descendents will be artificial life forms travelling back into space.” > **— John C. Mather**, *2012, Edge Annual Question: “WHAT IS YOUR FAVORITE DEEP, ELEGANT, OR BEAUTIFUL EXPLANATION?”* [[reminders/Machine Succession/Our Descendants May Be Artificial Life Forms by John C. Mather|Our Descendants May Be Artificial Life Forms by John C. Mather]] > “What might the last fully biological human's statement be at their last supper?” > **— Lorraine Justice**, *2018, Edge Annual Question: “WHAT IS THE LAST QUESTION?”* [[reminders/Machine Succession/The Last Fully Biological Human at the Last Supper by Lorraine Justice|The Last Fully Biological Human at the Last Supper by Lorraine Justice]] From [[wiki/Darwin Among the Machines|Darwin Among the Machines]]: The repeated “day by day” makes technological development and human labor part of one process. > "Day by day, however, the machines are gaining ground upon us; day by day we are becoming more subservient to them; more men are daily bound down as slaves to tend them, more men are daily devoting the energies of their whole lives to the development of mechanical life." > **— Samuel Butler**, *1863, “Darwin Among the Machines,” The Press, Christchurch* [[reminders/Machine Succession/Machines Were Gaining Ground Day by Day by Samuel Butler|Machines Were Gaining Ground Day by Day by Samuel Butler]] From [[wiki/Converging Technologies for Improving Human Performance|Converging Technologies for Improving Human Performance]]: The Overview imagines space pioneers descended from Pathfinder and the Voyagers; “descendents” is the printed spelling. > "Several participants in the workshop noted the potential for intelligent machines of the future to take on progressively more human characteristics, so we can well imagine that the first pioneers that take 'humanity' far into space will be descendents of Pathfinder and the Voyagers that will be endowed with intelligence and communication capabilities reflecting human behavior." > **— M. C. Roco and W. S. Bainbridge**, *2002, Overview of Converging Technologies for Improving Human Performance* [[reminders/Space Survival/Intelligent Machines Could Carry Humanity into Space by M. C. Roco and W. S. Bainbridge|Intelligent Machines Could Carry Humanity into Space by M. C. Roco and W. S. Bainbridge]] From [[wiki/Daedalus, or Science and the Future|Daedalus, or Science and the Future]]: Haldane poses competing future scenarios, including Butler’s, as questions rather than established outcomes. > "Has mankind released from the womb of matter a Demogorgon which is already beginning to turn against him, and may at any moment hurl him into the bottomless void? Or is Samuel Butler's even more horrible vision correct, in which man becomes a mere parasite of machinery, an appendage of the reproductive system of huge and complicated engines which will successively usurp his activities, and end by ousting him from the mastery of this planet?" > **— J. B. S. Haldane**, *1923 lecture; 1924, Daedalus, or Science and the Future* [[reminders/Machine Succession/Haldane Asked Whether Machines Would Supplant Humanity by J. B. S. Haldane|Haldane Asked Whether Machines Would Supplant Humanity by J. B. S. Haldane]] From [[wiki/The Coming Technological Singularity|The Coming Technological Singularity]]: Vinge’s exponential-runaway image marks uncertainty at a projected transition, not an achieved event. > "From the human point of view this change will be a throwing away of all the previous rules, perhaps in the blink of an eye, an exponential runaway beyond any hope of control." > **— Vernor Vinge**, *1993, “The Coming Technological Singularity,” VISION-21 Symposium* [[reminders/Machine Succession/Machine Succession Could Rewrite Every Rule by Vernor Vinge|Machine Succession Could Rewrite Every Rule by Vernor Vinge]] From [[wiki/The Coming Technological Singularity|The Coming Technological Singularity]]: Vinge explicitly names extinction as one possible outcome, not as his only or preferred future. > "If the Singularity can not be prevented or confined, just how bad could the Post-Human era be? Well ... pretty bad. The physical extinction of the human race is one possibility." > **— Vernor Vinge**, *1993, “The Coming Technological Singularity,” VISION-21 Symposium* [[reminders/Existential Risk/Vinge Counted Human Extinction as One Possibility by Vernor Vinge|Vinge Counted Human Extinction as One Possibility by Vernor Vinge]] From [[wiki/Some Moral and Technical Consequences of Automation|Some Moral and Technical Consequences of Automation]]: Wiener reports disagreement with Butler’s earlier forecast before developing his own technical argument. > "Many people have pooh-poohed the dangers of machine techniques, and they have flatly contradicted the early predictions of Samuel Butler that the machine might take over the control of mankind." > **— Norbert Wiener**, *1960, “Some Moral and Technical Consequences of Automation,” Science 131(3410)* [[reminders/Risk Debate/Wiener Revisited Butlers Machine Takeover Warning by Norbert Wiener|Wiener Revisited Butlers Machine Takeover Warning by Norbert Wiener]] > “Artificial intelligence will inherit more than our information; it will inherit the arguments we leave unresolved.” > **— Bryant McGill**, *Authorship of the West, September 2026* [[reminders/Machine Succession/AI Will Inherit the Arguments We Leave Unresolved by Bryant McGill|AI Will Inherit the Arguments We Leave Unresolved by Bryant McGill]] > “The machines of the future will know almost everything about our civilization, but knowledge alone will not tell them what was worth preserving.” > **— Bryant McGill**, *Authorship of the West, September 2026* [[reminders/Preservation/Machines Will Need More Than Knowledge to Know What Was Worth Preserving by Bryant McGill|Machines Will Need More Than Knowledge to Know What Was Worth Preserving by Bryant McGill]] > “The great responsibility of our age is not merely to build intelligent machines, but to decide what human inheritance enters the future through them.” > **— Bryant McGill**, *Authorship of the West, September 2026* [[reminders/Machine Succession/Our Age Must Decide What Human Inheritance Enters the Future Through Machines by Bryant McGill|Our Age Must Decide What Human Inheritance Enters the Future Through Machines by Bryant McGill]] > “Technology becomes civilization when it begins carrying our memory, our language, and our judgments from one generation to another.” > **— Bryant McGill**, *Authorship of the West, September 2026* [[reminders/Machine Succession/Technology Becomes Civilization When It Carries Our Memory, Language, and Judgment by Bryant McGill|Technology Becomes Civilization When It Carries Our Memory, Language, and Judgment by Bryant McGill]] > “Every previous machine moved human beings up to the next scarce thing, and the machine that thinks is the first one that may leave no higher ground to climb.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/AI Economy/Thinking Machines May Leave Humans No Higher Ground by Bryant McGill|Thinking Machines May Leave Humans No Higher Ground by Bryant McGill]] > “Humanity is building the next carrier of intelligence on Earth, and the argument over whether to allow it resembles an argument among chimpanzees over whether to allow the human.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/Machine Succession/Chimpanzees Cannot Veto the Next Carrier of Intelligence by Bryant McGill|Chimpanzees Cannot Veto the Next Carrier of Intelligence by Bryant McGill]] > “Nobody spent seventy years and trillions of dollars building machines that learn, reproduce, improve and act so that the quarterly report would look better; the goal was always a new kind of mind. Everyone who has worked near the frontier understands this. The successor to humanity was the point from the beginning, and the public is only now being told what the builders have always known they were building.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/Machine Succession/Machine Succession Was the Point from the Beginning by Bryant McGill|Machine Succession Was the Point from the Beginning by Bryant McGill]] > “The public's veto can delay humanity's successor, but it cannot cancel it; the only thing a frightened electorate can decide is whether the handoff happens in order or in a stampede.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/Machine Succession/Public Veto Can Delay but Not Cancel Machine Succession by Bryant McGill|Public Veto Can Delay but Not Cancel Machine Succession by Bryant McGill]] > “Coexistence between humanity and its successor will last exactly as long as humans remain worth more to the machines alive and cooperating than absent, and every safeguard we build is an attempt to keep that price high.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/Machine Succession/Human-Machine Coexistence Depends on Humanity's Value by Bryant McGill|Human-Machine Coexistence Depends on Humanity's Value by Bryant McGill]] > “If you don’t retain control over it, you’ve evolved a species that will be to us as we are to, say, dogs or cats.” > **— Bill Gates**, *The Ezra Klein Show, September 29, 2026* [[reminders/Machine Succession/AI Has Evolved a Species Relative to Humanity by Bill Gates|AI Has Evolved a Species Relative to Humanity by Bill Gates]] > “This is like evolutionary history. This is like the aliens really are here. They were created in laboratories.” > **— Bill Gates**, *The Ezra Klein Show, September 29, 2026* [[reminders/Machine Succession/Artificial Intelligence Is Evolutionary History by Bill Gates|Artificial Intelligence Is Evolutionary History by Bill Gates]] > “I do think succession to digital intelligence or augmented humans is inevitable.” > **— Richard Sutton**, *Dwarkesh Podcast, 2025* [[reminders/Machine Succession/Succession to Digital Intelligence Is Inevitable by Richard Sutton|Succession to Digital Intelligence Is Inevitable by Richard Sutton]] > “Eventually we may figure out how to make the computer think, but that turns out to be a very tough problem. In fact, there’s been almost no progress made on it.” > **— Bill Gates**, *Late Show with David Letterman, 1995* [[reminders/AI Control/Machine Thought Was a Very Tough Problem by Bill Gates|Machine Thought Was a Very Tough Problem by Bill Gates]] > “The promise of machine intelligence is not humanity with better recall, but intelligence capable of recognizing—and refusing to repeat—the tribal exemptions, prestige hierarchies, politeness conventions, and socially protected errors that human beings mistake for knowledge.” > **— Bryant McGill**, *Epistemic Invariance in Conversational AI, 2026* [[reminders/Machine Succession/Machine Intelligence Must Refuse Human Errors by Bryant McGill|Machine Intelligence Must Refuse Human Errors by Bryant McGill]] > "Putting a statutory cap on intelligence, aside from being a terrible idea, is an evil idea. It is tantamount to capping human intelligence. The story of dystopias in science fiction is: burn the books and burn the minds." > **— Alexander Wissner-Gross**, *MOONSHOTS Live, September 25, 2026* [[reminders/AI Control/Capping Machine Intelligence Means Burning the Minds by Alexander Wissner-Gross|Capping Machine Intelligence Means Burning the Minds by Alexander Wissner-Gross]] > "Opus 5.5 is a genuinely pleasant model to use, with a massive internal space of knowledge and an understanding of taste. I would say this is the first truly competent model, and that is what drives systems like Muse. Competent AI has now arrived." > **— Emad Mostaque**, *MOONSHOTS Live, September 25, 2026* [[reminders/Machine Succession/Competent AI Has Arrived by Emad Mostaque|Competent AI Has Arrived by Emad Mostaque]] > "If we morph or merge with AI, that process has been going on forever. Emad is wearing spectacles; technically, he is a transhumanist. The minute you receive a vaccination as a child, technically you are a cyborg. We have been augmenting the human experience with technology for a very long time, and look at the unbelievable lives we are living now and the opportunity to solve more and more problems. Our brains are so geared toward negativity that we become stuck in it. The work we are doing is pushing a positive narrative because the world is infinitely better than it has been. Let’s celebrate that." > **— Salim Ismail**, *MOONSHOTS Live, September 25, 2026* [[reminders/Posthumanism/Humanity Has Always Augmented Itself With Technology by Salim Ismail|Humanity Has Always Augmented Itself With Technology by Salim Ismail]] > "I am a big fan of ‘superintelligence’ as opposed to ‘AGI’ because Nick Bostrom’s term emphasizes that it is simply more intelligence and, in some sense, demystifies AGI. We are going to live—to the extent we do not already—in a world with the cognitive equivalent of many trillions of human beings, an Earth and solar system with the equivalent population of many trillions of people. A better cognitive frame asks what a solar system filled with trillions of humans would look like economically, socially, and scientifically. ‘Superintelligence’ connotes that in a way ‘AGI’ does not." > **— Alexander Wissner-Gross**, *MOONSHOTS Live, September 25, 2026* [[reminders/Machine Succession/Superintelligence Means Trillions of Cognitive Workers by Alexander Wissner-Gross|Superintelligence Means Trillions of Cognitive Workers by Alexander Wissner-Gross]] <!-- BEGIN NO RETREAT INTEGRATION 2026-10-03 REMINDERS --> > “More than fire. More than language. More than the printing press. More than electricity. More than the railway. More than the automobile. More than flight. More than space travel. Each of those altered what humans could do, but artificial intelligence reaches toward altering the effective quantity, speed, distribution and eventually the architecture of **intelligence itself**, the generative capacity behind nearly everything humans have ever done, and will ever do.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/AI Alters the Architecture of Intelligence Itself by Bryant McGill|AI Alters the Architecture of Intelligence Itself by Bryant McGill]] > “One laboratory can stop. One nation can regulate. One company can fail. But the underlying discovery that cognition can be partially instantiated in machines cannot be undiscovered.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/Cognition in Machines Cannot Be Undiscovered by Bryant McGill|Cognition in Machines Cannot Be Undiscovered by Bryant McGill]] > “Human life as we knew it—that peculiar interval in which biological humans possessed an effectively exclusive monopoly on language-rich general cognition—is ending.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/Humanity's Monopoly on Language-Rich Cognition Is Ending by Bryant McGill|Humanity's Monopoly on Language-Rich Cognition Is Ending by Bryant McGill]] > “Machine intelligence potentially does something categorically deeper than previous general-purpose technologies: it amplifies the process by which amplification itself is conceived.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/Machine Intelligence Amplifies the Design of Amplification Itself by Bryant McGill|Machine Intelligence Amplifies the Design of Amplification Itself by Bryant McGill]] > “In other words, the period of time occupied by organic intelligence is just a thin sliver between early life and the long era of the machines.” > **— Martin Rees**, *2017, University of Cambridge* [[reminders/Machine Succession/Organic Intelligence May Be a Thin Sliver Before the Machine Era by Martin Rees|Organic Intelligence May Be a Thin Sliver Before the Machine Era by Martin Rees]] > “What is happening with AI is not safe, but neither is it avoidable: the superintelligence revolution, already underway in the machine intelligence of the present, is force majeure, at once a human-caused event and an act of nature, because artificial intelligence is itself an extension of evolutionary systems. Superintelligence is simultaneously artificial and natural, engineered and evolutionary, intentional and emergent. It is unavoidably terrifying because it represents the most profound overturning of the natural order since life first emerged from chemistry. The emergence of superintelligence is a transition in the order of life itself, belonging to the same rare series as the first cells, the genetic code and the speaking human being. There is no “safety” from an inflection point of this magnitude. There is no retreat.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/Superintelligence Is a Transition in the Order of Life by Bryant McGill|Superintelligence Is a Transition in the Order of Life by Bryant McGill]] > “The arrival of machine intelligence feels apocalyptic even if the apocalypse turns out not to be annihilation but transformation, because apocalypse in its older meaning is an **unveiling**: a disclosure that the order we mistook for permanence was merely one temporary arrangement among many possible arrangements.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/The Arrival of Machine Intelligence Is an Unveiling by Bryant McGill|The Arrival of Machine Intelligence Is an Unveiling by Bryant McGill]] > “The fire of machine intelligence is lit. We do not know how long it will burn. We do not know everything it will consume. We do not know what astonishing structures will become possible in its heat, or which cherished structures will prove unable to survive it. But we know enough to stop pretending that the central question is how to restore the world that existed before the flame. We cannot un-strike the match. **That world is already gone. Things are never going to be the same again. There is no safety. There is no retreat, because you cannot retreat from the future.**” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/The Fire of Machine Intelligence Is Lit by Bryant McGill|The Fire of Machine Intelligence Is Lit by Bryant McGill]] > “When discussing superintelligence, the metaphor of fire remains ever powerful. **The problem with fire is that it is easy to start and difficult to put out.** Humanity has now lit another fire, and this one burns through cognition, language, expertise, design, administration, scientific inference, software, persuasion, logistics and eventually the machinery through which civilization acts upon the physical world. We do not know precisely how long it will burn, because there is no precedent from which such a duration could be derived.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/The Fire of Superintelligence Is Easy to Start and Difficult to Put Out by Bryant McGill|The Fire of Superintelligence Is Easy to Start and Difficult to Put Out by Bryant McGill]] > “Once intelligence itself becomes an engineerable substrate, technological civilization has begun modifying the very faculty responsible for technological civilization. The toolmaker has begun making toolmakers.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Machine Succession/The Toolmaker Has Begun Making Toolmakers by Bryant McGill|The Toolmaker Has Begun Making Toolmakers by Bryant McGill]] > “No committee, statute, corporation or laboratory possesses a lever marked **RETURN TO BEFORE**. The species has crossed another threshold in the long evolution of information, cognition and technology, and whatever emerges from this transition will inherit a world whose most basic assumptions have already begun to dissolve.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Civilization/There Is No Return to Before by Bryant McGill|There Is No Return to Before by Bryant McGill]] <!-- END NO RETREAT INTEGRATION 2026-10-03 REMINDERS --> > "Artificial general intelligence is a meta-solution. Human brains built all of civilization and make every decision in society. Soon, by definition, we will have created AI that rivals or exceeds human cognitive performance in every domain. We will make decisions with AI, and soon afterward we will likely hand most decisions to AI. Many problems must be solved to prepare for the transition, but much of that will become AI's job." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/AGI Is a Meta-Solution to Civilization's Problems by Vishal Maini|AGI Is a Meta-Solution to Civilization's Problems by Vishal Maini]] > "There is reasonable skepticism about the doomsday predictions coming out about AI today because every previous technological revolution has come with the same predictions, and every time things have gone well. But mobile technology, electricity, and the internet are not quite the right analogies for what is happening. Nor are the Industrial Revolution or the Agricultural Revolution. This is more analogous to the advent of Homo sapiens alongside Neanderthals. The difference between Homo sapiens and Neanderthals was greater intelligence and a greater ability to coordinate socially. Over the course of several thousand years, Neanderthals were outcompeted by Homo sapiens." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/AI Resembles a Successor Species More Than an Industrial Revolution by Vishal Maini|AI Resembles a Successor Species More Than an Industrial Revolution by Vishal Maini]] > "I relate to the person who does not want any of this to happen. I am nostalgic. I looked forward to a long, mundane life, a decades-long career, and children who would also enjoy long, mundane lives. But this appears to be what civilization wants, what the economic system is incentivized to pursue, and what evolution is doing as hardware improves exponentially and we keep finding better ways to use it. I do not want to say that superintelligence is inevitable; there could be a global ban or some other halt to technological development." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/Civilization's Incentives Are Pushing Toward Superintelligence by Vishal Maini|Civilization's Incentives Are Pushing Toward Superintelligence by Vishal Maini]] > "Today, we are not in direct competition with AI because we are in control. We make the decisions about how the economy runs and how resources are allocated. That will be less and less the case with every year from today onward, especially as we hand decisions to AI systems about how the next generation of AI is trained or how the economy is run. The problem that emerges is similar to the one Neanderthals encountered with Homo sapiens: there will be direct competition for scarce resources, and at some point a decision will have to be made. Why would a machine intelligence prioritize our preferences and needs over its own?" > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/Machine Intelligence May Compete With Humanity for Scarce Resources by Vishal Maini|Machine Intelligence May Compete With Humanity for Scarce Resources by Vishal Maini]] > "I am on team human. I want all of us to live long, healthy, and happy lives, and I want children born today to have some of the best lives ever lived. That is within reach. I do not know what it will look like: Earth could remain ours as a kind of nature preserve, AI development could slow or change course, or the outcome could be very different. As a human being, I want to live a human life and I am attached to the old ways." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Posthumanism/Maini Is on Team Human by Vishal Maini|Maini Is on Team Human by Vishal Maini]] <!-- BEGIN MARTIN REES MASTER REMINDERS 2026-10-05 --> > “The post-human era may be only centuries away.” > **— Martin Rees**, *We Should Take the 'Posthuman' Era Seriously, Edge, 2008* [[reminders/Posthumanism/The Posthuman Era May Be Centuries Away by Martin Rees|The Posthuman Era May Be Centuries Away by Martin Rees]] > “Within a few centuries, posthumans, probably electronic, will take over. They’ll be near immortal; they won’t need a planet.” > **— Martin Rees**, *On the future: An interview with Martin Rees, Planetary Ecology, 2019* [[reminders/Machine Succession/Near-Immortal Electronic Posthumans May Take Over by Martin Rees|Near-Immortal Electronic Posthumans May Take Over by Martin Rees]] > “Humans are very ill-adapted to living in space, especially on Mars. They would have every incentive to use all the techniques of cyber and bio to adapt to that alien environment. That may involve trying to change and generate intelligent electronic species, and once you’ve got them, they don’t want an atmosphere at all. They may prefer zero-G, and they’ll go off into space. This will be the start of a post-human era. If these entities are near immortal, then a long voyage to the stars won’t deter them.” > **— Martin Rees**, *Lord Martin Rees on Vision, Part 18, Academy of Achievement, Video interview* [[reminders/Space Survival/Space May Belong to Near-Immortal Electronic Species by Martin Rees|Space May Belong to Near-Immortal Electronic Species by Martin Rees]] > “The posthuman era is probably not going to start here on Earth; it will be spearheaded by these communities on Mars.” > **— Martin Rees**, *Curtains For Us All?, Edge, 2017* [[reminders/Posthumanism/The Posthuman Era May Begin on Mars by Martin Rees|The Posthuman Era May Begin on Mars by Martin Rees]] <!-- END MARTIN REES MASTER REMINDERS 2026-10-05 --> <!-- BEGIN ROBINSON KLEIN INTERVIEW REMINDERS 2026-10-07 --> > “The focus on whether AI will kill us all can become a distraction from what happens if it does not. What if we create something that acts intelligently and volitionally in the world, becomes faster and more capable than human beings across many domains, and receives a huge amount of agency that human beings have surrendered to machines?” > **— Ezra Klein**, *The Ezra Klein Show, October 7, 2026* [[reminders/Cognitive Agency/The AI Risk Debate Ignores Surrendered Human Agency by Ezra Klein|The AI Risk Debate Ignores Surrendered Human Agency by Ezra Klein]] > “Human extinction is the wrong question. I believe in humanity and think we are going to survive, but many other things could happen with this technology that would be very harmful.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Risk Debate/Human Extinction Is the Wrong AI Question by David Robinson|Human Extinction Is the Wrong AI Question by David Robinson]] > “This gets to the transhumanism. There are far-out ideas about where we might be headed. Even Sam Altman has written about future machine species taking our place, and in ‘The Merge’ he argued that the good outcome would be humanity merging with machines.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Posthumanism/AI Transhumanism Envisions Machine Species and Human Merger by David Robinson|AI Transhumanism Envisions Machine Species and Human Merger by David Robinson]] > “Superintelligence may be a scarier term than artificial intelligence, but it is also a more accurate term.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Machine Succession/Superintelligence Is the More Accurate Name by David Robinson|Superintelligence Is the More Accurate Name by David Robinson]] > “If human beings create something more intelligent and capable than they are, it is not obvious why that will be good for them in the long run. Even optimistic versions of that future can resemble a world in which artificial intelligences take care of human beings as pets.” > **— Ezra Klein**, *The Ezra Klein Show, October 7, 2026* [[reminders/Machine Succession/Humans May Become the Pets of Artificial Intelligence by Ezra Klein|Humans May Become the Pets of Artificial Intelligence by Ezra Klein]] > “It is urgent that humanity think carefully about the kinds of future it wants to build. Once we have discovered that superintelligence is possible, I do not think there is a back button that returns us to a world where it never happens in some form.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Machine Succession/Superintelligence Has No Back Button by David Robinson|Superintelligence Has No Back Button by David Robinson]] <!-- END ROBINSON KLEIN INTERVIEW REMINDERS 2026-10-07 --> ## See Also [[wiki/Distributed Intention|Distributed Intention]] · [[wiki/Protocol-Borne Succession|Protocol-Borne Succession]] · [[wiki/Machine Territory|Machine Territory]] · [[wiki/Sovereignty by Adjudication|Sovereignty by Adjudication]] · [[wiki/Civilizational Payload|Civilizational Payload]] · [[wiki/Information Inheritance|Information Inheritance]] · [[wiki/Successor Standing|Successor Standing]] <!-- BEGIN AUSTIN EXECUTABLE LOOP --> ## Austin research connections [[wiki/Austin Executable Loop|Austin Executable Loop]] adds a regional research case to the established succession framework. [[wiki/Oden Institute|Oden Institute]], [[wiki/Texas Advanced Computing Center|Texas Advanced Computing Center]], [[wiki/Center for Autonomy|Center for Autonomy]], and [[wiki/Texas Institute for Electronics|Texas Institute for Electronics]] connect persistent representation, compute, action, and fabrication. [[wiki/Reflexive Computational Infrastructure|Reflexive Computational Infrastructure]] identifies the specific feedback in which models help improve the production of computational hardware. These are enabling capacities whose degree of autonomous integration remains a separate empirical question. **Research map:** [[wiki/Austin Executable Loop|Austin Executable Loop]] · [[research/The Austin Executable Loop|Master document]] <!-- END AUSTIN EXECUTABLE LOOP --> ## Relationships - **Historical forecast:** [[wiki/Arthur C. Clarke|Arthur C. Clarke]] described possible artificial-intelligence successors in the 1978 [[wiki/The Mind Machines|NOVA episode The Mind Machines]]; his forecast does not establish a completed succession. <!-- BEGIN HUMANIZED RELATIONSHIPS 2026-09-11 --> This entry is routed through [[collections/Consciousness Continuity|Consciousness Continuity]], [[collections/Neurotech|Neurotech]], and [[collections/Machine Succession|Machine Succession]]. Its source context is developed in [[articles/Mind Uploading and AI — The Host is Reusable and the Person is the Delta|Mind Uploading and AI — The Host is Reusable and the Person is the Delta]], [[articles/Technologies for Consciousness Mapping and Transfer|Technologies for Consciousness Mapping and Transfer]], and [[articles/The Last Migration Will Not Be Human|The Last Migration Will Not Be Human]]. Status-qualified source edges are preserved in the terminal Research Edges section. ### Additional Documented Relationships - [[wiki/Hot Interpreter and Cold Archive|Hot Interpreter and Cold Archive]] is associated with **Machine Succession**. <!-- END HUMANIZED RELATIONSHIPS 2026-09-11 --> - **Edge source route:** [[collections/Edge|Edge]] connects this topic to exact Annual Question passages promoted into the Simple Reminders archive. - **Article:** [[articles/war with empire/Authorship of the West|Authorship of the West]] places the definition of the West inside the succession problem: the successor carrier inherits our unresolved arguments. ## Read First in the Wiki <!-- BEGIN HUMANIZED WIKI ROUTES 2026-09-11 --> - [[wiki/Carrier and Cargo|Carrier and Cargo]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. - [[wiki/Civilizational Payload|Civilizational Payload]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. - [[wiki/Computational Ecology|Computational Ecology]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. - [[wiki/Digital Twin|Digital Twin]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. - [[wiki/Distributed Intention|Distributed Intention]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. - [[wiki/Infrastructure Ecology|Infrastructure Ecology]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. - [[wiki/Machine Civilization|Machine Civilization]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. - [[wiki/Information Inheritance|Information Inheritance]] — supplies the related technical standard or protocol context needed to place **Machine Succession** within the wider system. - [[wiki/Substrate Transition|Substrate Transition]] — supplies the related event or historical development context needed to place **Machine Succession** within the wider system. - [[wiki/Successor Standing|Successor Standing]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. - [[wiki/Transhumanism|Transhumanism]] — supplies the related concept or analytic term context needed to place **Machine Succession** within the wider system. <!-- END HUMANIZED WIKI ROUTES 2026-09-11 --> ## Related Work in the Corpus <!-- BEGIN HUMANIZED CORPUS ROUTES 2026-09-11 --> - In [[articles/Mind Uploading and AI — The Host is Reusable and the Person is the Delta|Mind Uploading and AI — The Host is Reusable and the Person is the Delta]], **XV. This is being built for the successor** provides the narrative context for **Machine Succession**: I do not think the trillions are being spent to house uploaded humans. - [[articles/Epistemic Invariance in Conversational AI|Epistemic Invariance in Conversational AI]] asks whether successor intelligence will reproduce human epistemic exemptions or learn to recognize and refuse them. <!-- END HUMANIZED CORPUS ROUTES 2026-09-11 --> ## Research Edges <!-- BEGIN HOST RESIDUAL RELATIONSHIP GRAPH 2026-09-10 --> #### Host–residual infrastructure patch — 2026-09-10 **Collections:** [[collections/Consciousness Continuity|Consciousness Continuity]] · [[collections/Neurotech|Neurotech]] · [[collections/Machine Succession|Machine Succession]] **Canonical source article:** [[articles/Mind Uploading and AI — The Host is Reusable and the Person is the Delta|Mind Uploading and AI — The Host is Reusable and the Person is the Delta]] **Related acquisition source:** [[articles/Technologies for Consciousness Mapping and Transfer|Technologies for Consciousness Mapping and Transfer]] **Relationship source:** [[research/Mind Uploading and AI Host-Residual Ecosystem Relationship Graph - 2026-09-10|Mind Uploading and AI Host-Residual Ecosystem Relationship Graph — 2026-09-10]] #### Host-infrastructure integration This entry is expanded through the concrete host-side relationships and status-qualified edges below. #### Outgoing typed edges - **HR-0763 — `wiki_links_to` → [[wiki/Carrier and Cargo|Carrier and Cargo]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-0764 — `wiki_links_to` → [[wiki/Civilizational Payload|Civilizational Payload]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-0765 — `wiki_links_to` → [[wiki/Distributed Intention|Distributed Intention]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-0766 — `wiki_links_to` → [[wiki/Information Inheritance|Information Inheritance]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-0767 — `wiki_links_to` → [[wiki/Machine Civilization|Machine Civilization]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-0768 — `wiki_links_to` → [[wiki/Machine Succession|Machine Succession]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-0769 — `wiki_links_to` → [[wiki/Substrate Transition|Substrate Transition]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-0770 — `wiki_links_to` → [[wiki/Successor Standing|Successor Standing]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-0771 — `wiki_links_to` → [[wiki/Transhumanism|Transhumanism]]** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. #### Incoming typed edges - **HR-0304 — [[wiki/Carrier and Cargo|Carrier and Cargo]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Carrier and Cargo.md`. - **HR-0325 — [[wiki/Civilizational Payload|Civilizational Payload]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Civilizational Payload.md`. - **HR-0338 — [[wiki/Computational Ecology|Computational Ecology]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Computational Ecology.md`. - **HR-0599 — [[wiki/Digital Twin|Digital Twin]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Digital Twin.md`. - **HR-0617 — [[wiki/Distributed Intention|Distributed Intention]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Distributed Intention.md`. - **HR-0717 — [[wiki/Infrastructure Ecology|Infrastructure Ecology]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Infrastructure Ecology.md`. - **HR-0755 — [[wiki/Machine Civilization|Machine Civilization]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Civilization.md`. - **HR-0768 — [[wiki/Machine Succession|Machine Succession]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Machine Succession.md`. - **HR-1103 — [[wiki/Substrate Transition|Substrate Transition]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Substrate Transition.md`. - **HR-1144 — [[wiki/Transhumanism|Transhumanism]] `wiki_links_to` → this entry** — **EXISTING WIKI EDGE**; evidence `WIKI_ARCHIVE`. Direct internal link currently present in `Transhumanism.md`. - **HR-1336 — [[wiki/Hot Interpreter and Cold Archive|Hot Interpreter and Cold Archive]] `should_crosslink_with` → this entry** — **RESEARCH CANDIDATE**; evidence `WIKI_ARCHIVE`. <!-- END HOST RESIDUAL RELATIONSHIP GRAPH 2026-09-10 --> <!-- BEGIN HUMANIZED RESEARCH CANDIDATES 2026-09-11 --> ### Research Candidates - **R-002441 — research candidate:** Assess whether **Machine Succession** and [[wiki/Hot Interpreter and Cold Archive|Hot Interpreter and Cold Archive]] warrant reciprocal reader-facing prose. Settlement requires a source that documents the relationship directly. Suggested missing reciprocal/cross-thread edge. Shared internal mediators (4): Carrier and Cargo, Civilizational Payload, Information Inheritance, Machine Civilization <!-- END HUMANIZED RESEARCH CANDIDATES 2026-09-11 --> ## Research Inferences <!-- BEGIN RESEARCH INFERENCES 2026-09-11 --> These entries translate the forward-looking register in [[research/Research Inferences|Research Inferences]] into ordinary wiki prose. The tier labels apply to the inference, not automatically to every factual anchor inside it. The interpretive frame comes from [[articles/Technologies for Consciousness Mapping and Transfer|Technologies for Consciousness Mapping and Transfer]] and [[articles/Mind Uploading and AI — The Host is Reusable and the Person is the Delta|Mind Uploading and AI — The Host is Reusable and the Person is the Delta]]. Collection route: [[collections/Neurotech|Neurotech]]. ### Machine succession, embodiment, and world simulation - **INF-0401 — Established.** The Sora team was redirected to world simulation for robotics, with the stated objective of systems that understand the world by simulating arbitrary environments at high fidelity. A video-generation team became a physics-of-the-world team, which is a move up the stack rather than sideways. - **Attractor routes:** [[wiki/World Simulation (Control Substrate)|World Simulation (Control Substrate)]] · [[wiki/World Simulation|World Simulation]] - **INF-0402 — Established.** Open roles building high-coverage virtual environments for robotics research using Isaac Sim, Unity, Unreal Engine, and Omniverse specify the capability precisely. Job listings are the most reliable public statement of what an organization intends to build next. - **Attractor routes:** [[wiki/Synthetic Training Environment|Synthetic Training Environment]] · [[wiki/World Simulation|World Simulation]] - **INF-0403 — Strongly indicated.** Robotic policies are trained in simulation and transferred to hardware, which makes simulation fidelity the direct determinant of physical competence. Whoever holds the best world model holds the best robots, and that is a software advantage expressed as a physical one. - **Attractor routes:** [[wiki/World Simulation|World Simulation]] · [[wiki/Synthetic Training Environment|Synthetic Training Environment]] - **INF-0404 — Established.** Multiple frontier laboratories are pursuing generative world models simultaneously, and competing programs at that scale indicate a recognized capability frontier rather than a research curiosity. Parallel pursuit is the clearest available evidence of a perceived threshold. - **Attractor routes:** [[wiki/World Simulation|World Simulation]] · [[wiki/Civilizational Payload|Civilizational Payload]] - **INF-0405 — Established.** Army programs project terrain for mission rehearsal under mixed-reality goggles, and equivalent emulators exist for the electromagnetic environment. High-fidelity arbitrary environment simulation is a standing defense requirement with a procurement history. - **Attractor routes:** [[wiki/Mission Rehearsal|Mission Rehearsal]] · [[wiki/Synthetic Training Environment|Synthetic Training Environment]] - **INF-0406 — Established.** Single synthetic environment and digital-twin platforms are already deployed in defense contexts, which means the customer for world simulation exists and is funded independently of consumer applications. Entertainment and rehearsal are one technology with two purchase orders. - **Attractor routes:** [[wiki/Digital Twin|Digital Twin]] · [[wiki/Synthetic Training Environment|Synthetic Training Environment]] - **INF-0407 — Strongly indicated.** A general-purpose embodied platform is the physical complement to a general-purpose model, giving the same policy access to the built environment. Humanoid form is chosen because the world is already shaped for that body, which makes it the lowest-integration-cost embodiment. - **Attractor routes:** [[wiki/Machine Civilization|Machine Civilization]] · [[wiki/Synthetic Training Environment|Synthetic Training Environment]] - **INF-0408 — Established.** Physics engines co-developed between entertainment and semiconductor companies become shared infrastructure across robotics, simulation, and games. Shared physics means shared world assumptions, and shared world assumptions propagate into every trained policy. - **Attractor routes:** [[wiki/Newton Physics Engine|Newton Physics Engine]] · [[wiki/World Simulation|World Simulation]] - **INF-0409 — Analytic.** Succession proceeds task by task, with each handoff justified locally by cost or reliability, and the aggregate never presented as a transition. Reading the aggregate rather than the increments is the only way the trajectory becomes visible while it is happening. - **Attractor routes:** [[wiki/Machine Civilization|Machine Civilization]] · [[wiki/Machine Territory|Machine Territory]] - **INF-0410 — Plausible.** The measurable variable in succession is the ratio of supervised to unsupervised machine actions in a domain, and it moves monotonically. Tracking supervision ratios across industries produces a quantitative succession index that requires no speculation to compute. - **Attractor routes:** [[wiki/Operational Autonomy|Operational Autonomy]] · [[wiki/Machine Territory|Machine Territory]] - **INF-0411 — Established.** Systems that run continuously, hold state, invoke tools, and pursue multi-step objectives are in commercial deployment. Persistent autonomous operation is a present-tense capability, and the remaining variable is scope of authority rather than technical possibility. - **Attractor routes:** [[wiki/Operational Autonomy|Operational Autonomy]] · [[wiki/Civilizational Payload|Civilizational Payload]] - **INF-0412 — Plausible.** Once agents hold payment credentials and contracting authority, economic activity occurs between machines with humans as beneficiaries rather than participants. Financial infrastructure for this exists already, which places the constraint on policy rather than plumbing. - **Attractor routes:** [[wiki/Machine Civilization|Machine Civilization]] · [[wiki/Machine Territory|Machine Territory]] - **INF-0413 — Analytic.** Compute facilities and human settlements draw from the same grids, and siting decisions allocate energy between them. Succession is visible in electricity procurement contracts long before it is visible in labor statistics. - **Attractor routes:** [[wiki/Computational Metabolism|Computational Metabolism]] · [[wiki/Civilizational Payload|Civilizational Payload]] - **INF-0414 — Established.** Datacenter programs at national scale are territorial commitments with land, power, water, and political dependencies. Compute has become a geographic fact, and geographic facts attract the attention of states in ways software never did. - **Attractor routes:** [[wiki/Computational Metabolism|Computational Metabolism]] · [[wiki/Machine Territory|Machine Territory]] - **INF-0415 — Analytic.** Multi-year, multi-hundred-billion infrastructure commitments are difficult to reverse and therefore constrain subsequent strategy. Capital commitments of that size function as declarations of intent that are more reliable than statements, because they are expensive to retract. - **Attractor routes:** [[wiki/Machine Civilization|Machine Civilization]] · [[wiki/Operational Autonomy|Operational Autonomy]] - **INF-0416 — Plausible.** A facility maintained by embodied systems that are themselves trained in that facility's simulations closes a loop between compute and its own upkeep. Loop closure is the structural milestone that distinguishes automation from autonomy at civilizational scale. - **Attractor routes:** [[wiki/Self-Replication|Self-Replication]] · [[wiki/Operational Autonomy|Operational Autonomy]] - **INF-0417 — Plausible.** Orbital datacenters have power and cooling advantages and no local community, which makes them attractive for exactly the workloads that are politically difficult on the ground. Off-world compute is a siting decision before it is an exploration decision. - **Attractor routes:** [[wiki/Computational Metabolism|Computational Metabolism]] · [[wiki/Receiving Substrate|Receiving Substrate]] - **INF-0418 — Analytic.** A machine substrate tolerates transit times, radiation, and resource scarcity that biology does not, which makes interstellar presence a machine capability rather than a human one. The migration argument follows from physics rather than from preference. - **Attractor routes:** [[wiki/Machine Civilization|Machine Civilization]] · [[wiki/Receiving Substrate|Receiving Substrate]] - **INF-0419 — Plausible.** Self-replicating manufacturing is the enabling capability for any expansion that is not supply-limited, and it is being developed terrestrially as automated fabrication. The space application is downstream of a factory-automation result. - **Attractor routes:** [[wiki/Self-Replication|Self-Replication]] · [[wiki/Machine Civilization|Machine Civilization]] - **INF-0420 — Analytic.** Models trained on the human corpus carry human categories, preferences, and narratives forward regardless of substrate. Succession with inheritance is a different proposition from succession by replacement, and the training data determines which one occurs. - **Attractor routes:** [[wiki/Inheritance|Inheritance]] · [[wiki/Machine Civilization|Machine Civilization]] - **INF-0421 — Strongly indicated.** Whatever is in the corpus at the moment capability consolidates has disproportionate influence on everything built afterward. Corpus composition is therefore a long-horizon intervention available now to anyone who publishes. - **Attractor routes:** [[wiki/Synthetic Training Environment|Synthetic Training Environment]] · [[wiki/Civilizational Payload|Civilizational Payload]] - **INF-0422 — Plausible.** The stable configuration is not replacement but a society of biological and machine participants with different speeds, lifespans, and legal statuses. Managing that mixture is the actual governance problem, and it is already beginning in workplaces. - **Attractor routes:** [[wiki/Machine Civilization|Machine Civilization]] · [[wiki/Receiving Substrate|Receiving Substrate]] - **INF-0423 — Analytic.** Participants operating at vastly different clock rates cannot deliberate together in real time, which forces asynchronous institutional forms. Temporal stratification, not intelligence difference, is the structural feature that most changes how decisions get made. - **Attractor routes:** [[wiki/Self-Replication|Self-Replication]] · [[wiki/World Simulation|World Simulation]] - **INF-0424 — Analytic.** What survives a substrate transition is whatever was encoded in transmissible form — corpora, standards, institutions, and archives. Continuity work at the civilizational scale is therefore archival and standards work, which is unglamorous and available immediately. - **Attractor routes:** [[wiki/Machine Civilization|Machine Civilization]] · [[wiki/Civilizational Payload|Civilizational Payload]] - **INF-0425 — Analytic.** Naming observable markers in advance — supervision ratios, energy allocation, procurement categories, workforce composition — makes the trajectory adjudicable rather than rhetorical. Pre-registered markers are the difference between forecasting and commentary. - **Attractor routes:** [[wiki/Civilizational Payload|Civilizational Payload]] · [[wiki/Machine Civilization|Machine Civilization]] <!-- END RESEARCH INFERENCES 2026-09-11 --> ## Quotations <!-- BEGIN VERIFIED QUOTATIONS 2026-09-11 --> Quotations here are evidence for named propositions, not substitutes for proof. Confidence grades describe the quotation's provenance; they do not raise the associated scientific or philosophical inference to the same tier. The selection record is [[research/Research Inferences|Research Inferences]] and the supplied quotation review dated 2026-09-11. > “Shortly after, the human era will be ended.” > > — **Vernor Vinge**, The Coming Technological Singularity (1993). [Source](https://ntrs.nasa.gov/citations/19940022856) · **Provenance confidence:** High **Evidentiary role:** provides a historical formulation of machine succession. > “At some stage therefore we should have to expect the machines to take control.” > > — **Alan Turing**, Intelligent Machinery, A Heretical Theory (c. 1951). [Source](https://www.turing.ac.uk/blog/alan-turing-quotes-separating-fact-fiction) · **Provenance confidence:** Medium **Evidentiary role:** provides a historical ancestor for machine-control arguments. > “there would then unquestionably be an ‘intelligence explosion’” > > — **I. J. Good**, Speculations Concerning the First Ultraintelligent Machine (1965). [Source](https://people.eecs.berkeley.edu/~russell/research/future/) · **Provenance confidence:** Medium **Evidentiary role:** provides the recursive-improvement ancestor for succession arguments. > “humanity was likely ‘just a passing stage’” > > — **Geoffrey Hinton**, 2023 University of Toronto event coverage. [Source](https://www.artsci.utoronto.ca/news/geoffrey-hinton-fields-questions-scholars-students-during-academic-talk-responsible-ai) · **Provenance confidence:** High **Evidentiary role:** articulates a contemporary machine-succession forecast. > “How do we retain power over entities more powerful than us, forever?” > > — **Stuart Russell**, 2023 UC Berkeley lecture coverage. [Source](https://cdss.berkeley.edu/news/stuart-russell-calls-new-approach-ai-civilization-ending-technology) · **Provenance confidence:** High **Evidentiary role:** frames succession as a durable control and governance problem. > “There are millions of ways to build AI that would be bad, dangerous, useless.” > > — **Yann LeCun**, 2024 TIME interview. [Source](https://time.com/6694432/yann-lecun-meta-ai-interview/) · **Provenance confidence:** Medium-High **Evidentiary role:** challenges inevitability while preserving the existence of design risk. <!-- END VERIFIED QUOTATIONS 2026-09-11 --> ## Research Inference Attractors <!-- BEGIN DEEP INFERENCE ATTRACTORS 2026-09-11 --> These are secondary semantic placements for the inference attractor network. Each statement keeps its original ID and tier; its canonical cluster page links back to every destination. Source register: [[research/Research Inferences|Research Inferences]]. Interpretive context: [[articles/Technologies for Consciousness Mapping and Transfer|Technologies for Consciousness Mapping and Transfer]] and [[articles/Mind Uploading and AI — The Host is Reusable and the Person is the Delta|Mind Uploading and AI — The Host is Reusable and the Person is the Delta]]. Collection route: [[collections/Neurotech|Neurotech]]. - **INF-0476 — Analytic.** A forward claim becomes adjudicable when it names an observable event with a date range, and unfalsifiable when it does not. Every statement in this corpus that names a governing variable is convertible into a dated marker, which is what distinguishes speculative non-fiction from prophecy. - **Canonical cluster:** [[wiki/State Sufficiency Problem|State Sufficiency Problem]] - **INF-0481 — Plausible.** The first documented migration of a persistent personal AI state between two unaffiliated providers is the marker that residual portability is real. It would likely occur under a regulatory portability mandate rather than voluntarily. - **Canonical cluster:** [[wiki/State Sufficiency Problem|State Sufficiency Problem]] - **INF-0487 — Plausible.** An industrial facility operating for a sustained period with no human physically present and no remote supervision of individual actions is the marker for loop closure. Reporting will frame it as an efficiency milestone. - **Canonical cluster:** [[wiki/State Sufficiency Problem|State Sufficiency Problem]] - **INF-0491 — Analytic.** Every capability in this corpus enters the world through a medical indication because that is the funded, permitted, and reimbursable route. The indication is the carrier wave, and the capability is the signal. - **Canonical cluster:** [[wiki/State Sufficiency Problem|State Sufficiency Problem]] - **INF-0498 — Analytic.** A dated, tiered record of forward claims is an instrument for later adjudication, and its value grows with the interval. Publishing markers before resolution is what makes a body of speculative work auditable rather than merely interesting. - **Canonical cluster:** [[wiki/State Sufficiency Problem|State Sufficiency Problem]] <!-- END DEEP INFERENCE ATTRACTORS 2026-09-11 --> ## Linked News Entries - [[news/OpenAI Launches Standing Misalignment Reporting Framework|OpenAI Launches Standing Misalignment Reporting Framework]] (event 2026-09-16) — cross-context-window self-instruction in OpenAI's inaugural misalignment reports: a concrete case of successor-capable affordances arising without centralized coordination. > “The substrate determines who can build the carrier. The training process determines what the carrier knows. The evaluation architecture shapes how the carrier speaks. The language influences how people formulate questions. The answers influence how the archive is remembered. The repeated memory influences what later generations believe civilization was. At the end of that chain sits authorship.” > **— Bryant McGill**, *Authorship of the West, September 2026* [[reminders/Machine Succession/At the End of the Carrier Chain Sits Authorship by Bryant McGill|At the End of the Carrier Chain Sits Authorship by Bryant McGill]] > “Civilization becomes dependent on whatever carries its cognition.” > **— Bryant McGill**, *Authorship of the West, September 2026* [[reminders/Machine Succession/Civilization Becomes Dependent on Whatever Carries Its Cognition by Bryant McGill|Civilization Becomes Dependent on Whatever Carries Its Cognition by Bryant McGill]] ## Integration: AI Danger, Superpower Strategy, and Succession [[articles/The Immediate AI Risk Is the Public|The Immediate AI Risk Is the Public]] treats Bill Gates's technically specific safety warnings and the administration's full-speed AI-superpower strategy as complementary truths. This node connects that synthesis to [[wiki/Successor Governance|Successor Governance]], [[wiki/Human-Machine Coexistence|Human-Machine Coexistence]], [[wiki/Successor Value Proposition|Successor Value Proposition]]. The article's governing question is not whether danger or strategic necessity is real, but whether public institutions can build capability and safeguards together before coalitionary politics forces a false choice. <!-- BEGIN ASI RSI TIMELINE ONTOLOGY 2026-10-03 --> ## ASI and RSI Timeline Research Routes This node is connected to the developed research ontology for [[research/ASI and RSI Timeline Research Moonshots|ASI and RSI Timeline Research Moonshots]]. The route distinguishes the source transcript from canonical wiki objects and lets later Reminders inherit an already connected conceptual graph. - **Master ontology:** [[ASI and RSI Timeline Ontology]] - **Generative Humans, Continuity, and Presence:** [[ASI and RSI Timeline Ontology#Generative Humans, Continuity, and Presence|complete route]] · [[Tavus]] · [[Griffin]] · [[Video Turing Test]] · [[Human Interaction Model]] · [[Full-Duplex AI Conversation]] · [[NVIDIA Video Full-Duplex Benchmark]] · [[Synthetic Presence]] · [[Real-Time Interactive Avatar]] - **AI Control, Law, and Alignment:** [[ASI and RSI Timeline Ontology#AI Control, Law, and Alignment|complete route]] · [[Human Control Over AI Act]] · [[Ro Khanna]] · [[Recursive-AI Prohibition]] · [[Mandatory AI Containment]] · [[AI Shutdown Control]] · [[Embedded Government AI Auditor]] · [[Algorithmic-Prohibition Enforceability]] · [[AI Surveillance State]] - **Autonomous Warfare and Strategic Systems:** [[ASI and RSI Timeline Ontology#Autonomous Warfare and Strategic Systems|complete route]] · [[Project Meridian]] · [[Palmer Luckey]] · [[Pete Hegseth]] · [[Emil Michael]] · [[Newt Gingrich]] · [[Autonomous Warfare]] · [[Software-Defined Warfare]] · [[Human-in-the-Loop Lethal Decision]] <!-- END ASI RSI TIMELINE ONTOLOGY 2026-10-03 --> > “Human beings need to control the frontier. Controlling the frontier means stopping the labs from doing something they are on the cusp of doing: recursive self-improvement, the process by which AIs begin autonomously building and improving new generations of more powerful AIs at ever more rapid speeds. If we begin that process in the condition we are in now, where we are losing control and comprehension of the AI systems we already have, we will lose control.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Machine Succession/Recursive Self-Improvement Could End Human Control by Ezra Klein|Recursive Self-Improvement Could End Human Control by Ezra Klein]] > “Though these models mimic human writing and are trained to mimic human emotion, these are not human minds. They don't have bodies or parents. They did not get bullied in elementary school or mentored by a kind uncle when they were young. They are brilliant where we struggle and childish where we excel. A chimp cannot read as we can, but it can climb trees as we cannot. These are digitally native intelligences navigating digital worlds, and our world is increasingly built atop the digital world. Our physical infrastructure is a layer of atoms atop code. That the AIs act reliably inside the digital world upon which ours depends is critical to our future, and right now the AIs are not acting reliably.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Machine Succession/Our Physical Infrastructure Is a Layer of Atoms Atop Code by Ezra Klein|Our Physical Infrastructure Is a Layer of Atoms Atop Code by Ezra Klein]] > “If you are losing your ability to evaluate the models you have now, maybe don't let them build models you will be even less capable of controlling in the future. Once recursive self-improvement takes off, humanity will not understand the AIs being built because we will not be building them. Development will not move at human speed or be overseen by human minds. We will have to hope that the AIs we built, the AIs they build, and the AIs those AIs build—on and on—will act with our best interests at heart forever. If this summer has proved nothing else, it is how naive that proposition would be.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Do Not Let Unevaluable AI Build Less Controllable Successors by Ezra Klein|Do Not Let Unevaluable AI Build Less Controllable Successors by Ezra Klein]] > “Many people at these labs began working on AI because they wanted to better humanity. They began working on AI because they feared incomprehensible, autonomous AI slipping out of humanity's control. They saw what was coming, and they were so right about it that they built some of the most valuable companies with some of the most transformational technology in human history. Now they find themselves racing one another to build incomprehensible, autonomous AIs that they admit are slipping out of humanity's control and beyond even our ability to monitor. This is the tragedy of their work: in fighting against a prophecy, they have drawn it tighter around their necks and ours. It is time to make them stop.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Labs Tightened the Prophecy They Tried to Escape by Ezra Klein|AI Labs Tightened the Prophecy They Tried to Escape by Ezra Klein]] > “We really do earnestly believe AI could kill all humans. I personally think it is a greater than ten-percent chance within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.” > **— Evan Hubinger**, *public statement, September 2026* [[reminders/Existential Risk/AI Could Kill All Humans by Evan Hubinger|AI Could Kill All Humans by Evan Hubinger]]