# Superintelligence **Domain:** Artificial Intelligence / Governance **Doc Type:** Concept Node **Maturity:** Evolving **Related:** [[Superintelligence Cradle Theory]], [[Distributed Intelligence]], [[Algorithmic Governance]], [[Rights]] --- ## Definition **Superintelligence** is intelligence whose general or strategically relevant capabilities substantially exceed those of humans or human institutions. ## Corpus Context Within this corpus the important question is architectural: whether such capacity appears in one model, a distributed [[Superorganism|superorganism]], or a human–machine institutional ecology. Capability does not automatically confer jurisdiction. ## See Also [[Superintelligence Cradle Theory]], [[Distributed Intelligence]], [[Algorithmic Governance]], [[Rights]] ## Simple Reminders, Quotations, and Thoughts > "Experts call a machine that can "think" a General Artificial Intelligence." > **— Michael Vassar**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Machine Succession/A Thinking Machine Is Artificial General Intelligence by Michael Vassar|A Thinking Machine Is Artificial General Intelligence by Michael Vassar]] > "We must not grant autonomy to systems that we do not understand and that we cannot control." > **— Thomas G Dietterich**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/AI Control/Never Grant Autonomy to Systems We Cannot Control by Thomas G Dietterich|Never Grant Autonomy to Systems We Cannot Control by Thomas G Dietterich]] > "Maybe, if you do work on AI, our superintelligent machine overlords will be good to you." > **— Antony Garrett Lisi**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Machine Succession/Superintelligent Machines May Favor Their Builders by Antony Garrett Lisi|Superintelligent Machines May Favor Their Builders by Antony Garrett Lisi]] > "Techno-optimists believe that progress is near a singularity, the hypothetical moment when machines will reach the point of a greater-than-human intelligence." > **— Satyajit Das**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Machine Succession/The Singularity Is a Hypothesis About Greater-Than-Human Intelligence by Satyajit Das|The Singularity Is a Hypothesis About Greater-Than-Human Intelligence by Satyajit Das]] > "The Singularity—the fateful moment when AI surpasses its creators in intelligence and takes over the world—is a meme worth pondering." > **— Daniel C. Dennett**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Machine Succession/The Singularity Is a Meme Worth Examining by Daniel C. Dennett|The Singularity Is a Meme Worth Examining by Daniel C. Dennett]] > "Machines will have several advantages: most obviously, faster processing speed — an artificial neuron can operate a million times faster than its biological counterpart. Machine intelligences may also have superior computational architectures and learning algorithms. These "qualitative" advantages, while harder to predict, may be even more important than the advantages in processing power and memory capacity. Furthermore, artificial intellects can be easily copied, and each new copy can — unlike humans — start life fully-fledged and endowed with all the knowledge accumulated by its predecessors. Given these considerations, it is possible that one day we may be able to create "superintelligence": a general intelligence that vastly outperforms the best human brains in every significant cognitive domain." > **— Nick Bostrom**, *2009, Edge Annual Question, “What Will Change Everything?”* [[reminders/Machine Succession/Machines Can Inherit Everything Their Predecessors Knew by Nick Bostrom|Machines Can Inherit Everything Their Predecessors Knew by Nick Bostrom]] > “My optimism about skeuomorphs arises, as optimism often does, from former pessimism. I grew up with Alan Turing's unsettling vision of a future machine indistinguishable from a human in its reactions. Ray Kurzweil's provocative prediction of the impending 'singularity'—the point when computer intelligence would start to leave humans gasping in its intellectual wake—added to my fears. I actually began to worry that efforts to enculture my children with Shakespeare and Darwin, and even with spiritual and moral values, might be rendered peremptorily redundant by cold robotic Übermenschen.” > **— Timothy Taylor**, *2007, Edge Annual Question, “What Are You Optimistic About?”* [[reminders/Machine Succession/Children May Adapt to Machine Intelligence Better Than Their Parents by Timothy Taylor|Children May Adapt to Machine Intelligence Better Than Their Parents by Timothy Taylor]] > “I'm 62, so I'll have to limit my projections to what I expect to happen in the next two to three decades. I believe these will be the most interesting times in human history (Remember the old Chinese curse about "interesting times?") Humanity will see, before I die, the "Singularity," the day when we finally create a human level artificial intelligence. This involves considering the physics advances that will be required to create the computer that is capable of running a strong AI program.” > **— Frank Tipler**, *2009, Edge Annual Question, “What Will Change Everything?”* [[reminders/Machine Succession/Posthuman Evolution Will Change Us All by Frank Tipler|Posthuman Evolution Will Change Us All by Frank Tipler]] > “The publication in 2015 of a new paper from Leicester University, "The Anthropocene Biosphere," provides more evidence that the changes wrought upon the climate by human civilization are set to bring down a sixth mass extinction. According to the paper’s co-author, geologist Peter Haff, we have already entered a period of fundamental changes to the planetary system that may continue to change the world beyond our imagination, a fear echoed by Adelaide University’s John Long in the New Scientist last month. All of us can provide anecdotal evidence of the shifts that are changing our environment. Today I received a call from a friend in Engadin, Switzerland, where Nietzsche wrote Thus Spoke Zarathustra. In late December, at an altitude of 2,000 metres, there is no snow. In Hyde Park, the daffodils are in bloom.” > **— Hans Ulrich Obrist**, *2016, Edge Annual Question, “What Do You Consider The Most Interesting Recent [Scientific] News? What Makes It Important?”* [[reminders/Existential Risk/The Sixth Mass Extinction Demands Immediate Action by Hans Ulrich Obrist|The Sixth Mass Extinction Demands Immediate Action by Hans Ulrich Obrist]] > “Cynics would go further, and say that anything that is scientifically and technically possible will be done — somewhere, sometime — despite ethical and prudential objections, and whatever the regulatory regime. Whether this idea is true or false, it's an exceedingly dangerous one, because it's engenders despairing pessimism, and demotivates efforts to secure a safer and fairer world. The future will best be safeguarded — and science has the best chance of being applied optimally — through the efforts of people who are less fatalistic.” > **— Martin Rees**, *2006, Edge Annual Question, “WHAT IS YOUR DANGEROUS IDEA?”* [[reminders/Existential Risk/Anything Technically Possible May Be Done Somewhere Sometime by Martin Rees|Anything Technically Possible May Be Done Somewhere Sometime by Martin Rees]] > “Can we program a computer to find a 10,000-bit string that encodes more actionable wisdom than any human has ever expressed?” > **— Scott Aaronson**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Machine Succession/Can a Computer Encode More Actionable Wisdom Than Any Human by Scott Aaronson|Can a Computer Encode More Actionable Wisdom Than Any Human by Scott Aaronson]] > “Is the unipolar future of a "singleton" the inevitable destiny of intelligent life?” > **— Brian Christian**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Machine Succession/Is a Singleton the Inevitable Destiny of Intelligent Life by Brian Christian|Is a Singleton the Inevitable Destiny of Intelligent Life by Brian Christian]] > “The central issue is the social, not scientific, definition of "thinking". A generation of Western intellectuals who took their identity mainly from their intelligence has grown too old to ask the question with any conviction, and anyway, machines are all around them thinking up a storm.” > **— Pamela McCorduck**, *2001, Edge Annual Question, “WHAT QUESTIONS HAVE DISAPPEARED?”* [[reminders/Machine Succession/Machines Are Already Thinking Up a Storm by Pamela McCorduck|Machines Are Already Thinking Up a Storm by Pamela McCorduck]] > “How much time will pass between the last minute before artificial superintelligence and the first minute after it?” > **— Bruno Giussani**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Machine Succession/The Minute Before and After Artificial Superintelligence by Bruno Giussani|The Minute Before and After Artificial Superintelligence by Bruno Giussani]] > “What will be the literally last question that will preoccupy future superintelligent cosmic life for as long as the laws of physics permit?” > **— Max Tegmark**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Deep Time/What Will Superintelligent Cosmic Life Ask Last by Max Tegmark|What Will Superintelligent Cosmic Life Ask Last by Max Tegmark]] > “When will "human being" cease to be a meaningful category to speak of?” > **— James J. O'Donnell**, *2018, Edge Annual Question, “WHAT IS THE LAST QUESTION?”* [[reminders/Personhood/When Will Human Being Stop Being a Meaningful Category by James J. O'Donnell|When Will Human Being Stop Being a Meaningful Category by James J. O'Donnell]] > “When we establish contact with such an intelligence, will we receive instructions for building a machine to upload Jodie Foster? Probably not. The download will proceed the other way.” > **— George Dyson**, *1999, Edge Annual Question: “WHAT IS THE MOST IMPORTANT INVENTION IN THE PAST TWO THOUSAND YEARS?”* [[reminders/Machine Succession/The Download Will Proceed the Other Way by George Dyson|The Download Will Proceed the Other Way by George Dyson]] > “The most dangerous idea is the genetic breakthrough that made people capable of ideas themselves. The idea of ideas is nice enough in principle; and ideas certainly have had their impact for good. But one of these days one of those nice ideas is likely to have the unintended consequence of destroying everything we know.” > **— Seth Lloyd**, *2006, Edge Annual Question: “WHAT IS YOUR DANGEROUS IDEA?”* [[reminders/Existential Risk/The Most Dangerous Idea Was the Capacity for Ideas by Seth Lloyd|The Most Dangerous Idea Was the Capacity for Ideas by Seth Lloyd]] > “For the first time in history — as far as we know! — we humans live in a world that we could completely destroy.” > **— Howard Gardner**, *2006, Edge Annual Question: “WHAT IS YOUR DANGEROUS IDEA?”* [[reminders/Existential Risk/We Are the First Humans Who Could Destroy the Whole World by Howard Gardner|We Are the First Humans Who Could Destroy the Whole World by Howard Gardner]] > “So here we are in 2012, with people writing essays and wondering whether their descendents will be artificial life forms travelling back into space.” > **— John C. Mather**, *2012, Edge Annual Question: “WHAT IS YOUR FAVORITE DEEP, ELEGANT, OR BEAUTIFUL EXPLANATION?”* [[reminders/Machine Succession/Our Descendants May Be Artificial Life Forms by John C. Mather|Our Descendants May Be Artificial Life Forms by John C. Mather]] > “What might the last fully biological human's statement be at their last supper?” > **— Lorraine Justice**, *2018, Edge Annual Question: “WHAT IS THE LAST QUESTION?”* [[reminders/Machine Succession/The Last Fully Biological Human at the Last Supper by Lorraine Justice|The Last Fully Biological Human at the Last Supper by Lorraine Justice]] > “Can rational beings such as Bayesian robots, humans, super-intelligent AIs ever reach agreement?” > **— Frank Tipler**, *2018 Edge Annual Question, question* [[reminders/Machine Succession/Can Humans and Superintelligent AIs Ever Agree by Frank Tipler|Can Humans and Superintelligent AIs Ever Agree by Frank Tipler]] > “Will the behavior of a superintelligent AI be mostly determined by the results of its reasoning about the other superintelligent AIs?” > **— Jaan Tallinn**, *2018 Edge Annual Question, question* [[reminders/Machine Succession/Will Superintelligent AIs Be Shaped by Other AIs by Jaan Tallinn|Will Superintelligent AIs Be Shaped by Other AIs by Jaan Tallinn]] ## Relationships - **Edge Annual Question source relationship:** [[collections/Edge|Edge]] preserves the annual-question source corpus from which a proposition-specific quotation is connected to this entry. - **Edge source route:** [[collections/Edge|Edge]] connects this topic to exact Annual Question passages promoted into the Simple Reminders archive. ## Sources / Provenance - Developed through [[articles/Beyond the Sea|Beyond the Sea]] and the linked corpus.