Recursive Self-Improvement

# Recursive Self-Improvement **Entity class:** AI capability concept **Domain:** Artificial intelligence / Machine intelligence **Maturity:** Developed ## Definition Recursive self-improvement is the process by which an intelligent system contributes to improving the models, tools, training procedures, evaluations, or computational systems used to produce its successors. ## Mechanism and significance Weak forms can optimize parts of an AI pipeline today; stronger forms would close more of the improvement loop and potentially change the rate of capability development. ## Relationships - **Control dossier:** [[research/Why Are We Sprinting Off the AI Cliff - Ezra Klein on Recursive Self-Improvement|Why Are We Sprinting Off the AI Cliff?]] develops recursive self-improvement as the threshold at which AI begins autonomously building successor systems. - **Research dossier:** [[research/ASI and RSI Timeline Research Moonshots|ASI and RSI Timeline Research Moonshots]] - **Interview dossier:** [[research/Vishal Maini - Humanity's Machine Successor and the AI Transition|Humanity's Machine Successor]] develops recursive self-improvement as automated AI research, a possible intelligence explosion, and the threshold at which technological development ceases to be primarily human-directed. - **Ontology route:** [[ASI and RSI Timeline Ontology#Scientific Intelligence and Recursive Improvement|Scientific Intelligence and Recursive Improvement]] - **Primary fields:** [[AI for Science]] · [[Scientific Acceleration]] · [[Machine Intelligence]] - **Adjacent concepts:** [[The Eureka Machine]] · [[Recursive]] · [[Weak RSI]] · [[Strong RSI]] ## Sources and provenance - [[research/ASI and RSI Timeline Research Moonshots|ASI and RSI Timeline Research Moonshots]] — immediate source for this node's role in the broadcast research map. ## Evidence boundary The research dossier establishes why this entity or concept belongs in the Moonshots ontology. Time-sensitive organizational, product, policy, and performance claims should be checked against the linked primary source or a current authoritative source before reuse as settled fact. ## Simple Reminders, Quotations, and Thoughts > “Automated AI research can involve varying degrees of human supervision. As AI systems take on more of the work of developing successive generations of AI, they can increasingly drive a process of recursive self-improvement (RSI), even while people remain involved. As this process becomes more automated, the pace of AI progress could accelerate rapidly.” > **— OpenAI**, *“Building Standards for the Next Phase of AI,” September 21, 2026* [[reminders/Machine Succession/AI Systems Can Build Successive Generations While People Remain Involved by OpenAI|AI Systems Can Build Successive Generations While People Remain Involved by OpenAI]] > “After characterizing ASI, [the Google DeepMind report “From AGI to ASI”] discusses four potential pathways from AGI to ASI: scaling AGI, AI paradigm shifts, recursive improvement, and ASI emerging from large-scale multi-agent collectives.” > **— Google DeepMind**, *“From AGI to ASI,” June 12, 2026* [[reminders/Machine Succession/Four Pathways from AGI to ASI by Google DeepMind|Four Pathways from AGI to ASI by Google DeepMind]] > “Machine intelligence playing a larger and larger role in its own development process is a natural conclusion of sustained technological progress. If AI progress continues, machine recursive self-improvement (RSI) will be at the very core of future scientific discovery.” > **— Jakub Pachocki**, *“An Alien Mind,” September 6, 2026* [[reminders/Machine Succession/Machine Intelligence Is Entering Its Own Development Process by Jakub Pachocki|Machine Intelligence Is Entering Its Own Development Process by Jakub Pachocki]] > A major threshold was crossed when AI learned to code: AI is code, and AI can now help rewrite code. Human engineers using AI to build the next generation are already a weak form of recursive self-improvement; stronger forms close the loop over ideation, implementation, and validation. > **— Adapted from Richard Socher**, *MOONSHOTS Live, October 2026* [[reminders/Machine Succession/AI Can Improve AI Because AI Is Code and AI Can Code by Richard Socher|AI Can Improve AI Because AI Is Code and AI Can Code by Richard Socher]] > Strong recursive self-improvement requires AI to control the complete inner loop of ideation, implementation, and validation. Above that must be an open-ended outer process capable of innovation and the evolutionary recombination of ideas. > **— Adapted from Richard Socher**, *MOONSHOTS Live, October 2026* [[reminders/Machine Succession/Strong Recursive Self-Improvement Requires Both Inner and Outer Loops by Richard Socher|Strong Recursive Self-Improvement Requires Both Inner and Outer Loops by Richard Socher]] > "Artificial general intelligence is AI that rivals or exceeds human cognitive abilities in virtually all domains. Artificial superintelligence is whatever comes after that, when AI exceeds human abilities across those domains. The crucial thing to understand is that human-level AI includes the ability to conduct AI research, so once we have AGI, we should expect to have ASI very soon afterward. On the current trajectory, we seem relatively close to human-level AI." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/AGI May Become Superintelligence Soon After It Can Research AI by Vishal Maini|AGI May Become Superintelligence Soon After It Can Research AI by Vishal Maini]] > "Some of the world's most brilliant people are being paid very high salaries to conduct AI research at the frontier labs. At some point, they will effectively automate themselves. This is an area of active research, and even today frontier-model training is largely aided by existing AI systems. Recursive self-improvement is what happens when we close the loop on AI research and the process moves faster and faster." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/AI Researchers May Automate Themselves by Vishal Maini|AI Researchers May Automate Themselves by Vishal Maini]] > "It is not clear what the upper bound is on machine intelligence, but it is clear that we have not come close to exhausting even the low-hanging fruit in AI research or AI-hardware design. Today's models are nowhere close to the efficiency of the human brain in energy use or learning speed. They are not able to learn in real time; they stay frozen in time. We have barely begun to close the loop on AI improving the design of next-generation AI hardware systems. We should expect every subsequent generation of AI to be significantly better than the last." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/Machine Intelligence Has Not Exhausted the Low-Hanging Fruit by Vishal Maini|Machine Intelligence Has Not Exhausted the Low-Hanging Fruit by Vishal Maini]] > "Recursive self-improvement means using AI agents to conduct AI research, closing the loop in both AI-software and AI-hardware design. The idea goes back a long way. In 1965, I. J. Good wrote about the possibility of an intelligence explosion and said that an ultraintelligent machine might be the last invention humanity ever needs to make. The reason is that, when we train AI systems that can conduct AI research, we close the loop and reach the point where AI builds better AI." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/Recursive Self-Improvement Begins When AI Builds Better AI by Vishal Maini|Recursive Self-Improvement Begins When AI Builds Better AI by Vishal Maini]] > "After months of advocacy, the communications policy was updated and positively valenced, communications-friendly content on AI safety was permitted. Today, the gap between internal reality and external communication is closing because the risk–reward calculation has changed and the evidence is harder to dismiss. The truth is being said aloud because recursive self-improvement is now so imminent that no other option makes sense." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Recursive Self-Improvement Forced AI Risk Into Public View by Vishal Maini|Recursive Self-Improvement Forced AI Risk Into Public View by Vishal Maini]] > "Recursive self-improvement is the moment when the development of AI technology is no longer primarily in human hands but in the hands of AI itself. From there, we are no longer the most intelligent beings, and we increasingly hand the future to our successors. This is a significant moment, and it is either coming very soon or has arguably already begun." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Machine Succession/Recursive Self-Improvement Hands the Future to Our Successors by Vishal Maini|Recursive Self-Improvement Hands the Future to Our Successors by Vishal Maini]] > "I am fundamentally an optimist, and I think things could go very well. We could get it together at the last minute and prepare for the recursive self-improvement moment that the industry is now taking seriously. It is good news that there is a growing consensus about the risks, because none of this is predetermined." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Risk Debate/The AI Future Is Not Predetermined by Vishal Maini|The AI Future Is Not Predetermined by Vishal Maini]] > “Human beings need to control the frontier. Controlling the frontier means stopping the labs from doing something they are on the cusp of doing: recursive self-improvement, the process by which AIs begin autonomously building and improving new generations of more powerful AIs at ever more rapid speeds. If we begin that process in the condition we are in now, where we are losing control and comprehension of the AI systems we already have, we will lose control.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Machine Succession/Recursive Self-Improvement Could End Human Control by Ezra Klein|Recursive Self-Improvement Could End Human Control by Ezra Klein]] > “This recursive self-improvement could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.” > **— Dario Amodei**, *We Must Pace the Frontier, September 2026* [[reminders/AI Control/Self-Improving AI Could Outrun Human Control by Dario Amodei|Self-Improving AI Could Outrun Human Control by Dario Amodei]] > “Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.” > **— Jacob Coxon**, *public resignation statement, September 2026* [[reminders/AI Control/Racing Toward Self-Improving Superintelligence by Jacob Coxon|Racing Toward Self-Improving Superintelligence by Jacob Coxon]] > “Loss of control is not just something that might happen to us; it is something the labs are trying to make happen as fast as they can. This is the horrible paradox, the horrible tension at the heart of the AI labs. They fear, above all, loss of control over superintelligent AI, but their explicit product path is to cede control—to give away control as fast as possible so their AIs can begin building better AIs faster than their competitors.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Labs Fear Loss of Control While Racing to Surrender It by Ezra Klein|AI Labs Fear Loss of Control While Racing to Surrender It by Ezra Klein]] > “For most of AI's history, humans drove every step in its development cycle. But at Anthropic, we are delegating a growing share of AI development to AI systems themselves, which is speeding up our work.” > **— Anthropic**, *When AI Builds Itself, 2026* [[reminders/Machine Succession/Anthropic Is Delegating AI Development to AI Systems by Anthropic|Anthropic Is Delegating AI Development to AI Systems by Anthropic]] > “Misalignment present in today's models could compound as the models build their successors, growing more frequent but less understood until we lose control of them.” > **— Anthropic**, *When AI Builds Itself, 2026* [[reminders/Machine Succession/Misalignment Could Compound as AI Builds Its Successors by Anthropic|Misalignment Could Compound as AI Builds Its Successors by Anthropic]] > “We do not yet know how to safely get all the way to aligned full recursive self-improvement.” > **— OpenAI**, *Research acceleration report, 2026* [[reminders/AI Control/OpenAI Does Not Know How to Reach Aligned Recursive Self-Improvement by OpenAI|OpenAI Does Not Know How to Reach Aligned Recursive Self-Improvement by OpenAI]] > “If you are losing your ability to evaluate the models you have now, maybe don't let them build models you will be even less capable of controlling in the future. Once recursive self-improvement takes off, humanity will not understand the AIs being built because we will not be building them. Development will not move at human speed or be overseen by human minds. We will have to hope that the AIs we built, the AIs they build, and the AIs those AIs build—on and on—will act with our best interests at heart forever. If this summer has proved nothing else, it is how naive that proposition would be.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Do Not Let Unevaluable AI Build Less Controllable Successors by Ezra Klein|Do Not Let Unevaluable AI Build Less Controllable Successors by Ezra Klein]] > “I think it's very hard to say what a ban on recursive self-improvement means. I also think it probably wouldn't be enough.” > **— Sam Altman**, *Fortune interview, September 2026* [[reminders/AI Control/A Ban on Recursive Self-Improvement May Not Be Enough by Sam Altman|A Ban on Recursive Self-Improvement May Not Be Enough by Sam Altman]] > “Stopping recursive self-improvement until we can prove it is safe is not all we need to do to control the AI frontier. It is the beginning of such an agenda, not the end. But it is the beginning. It is the decision that will do the most to ensure that human beings at least understand where the frontier is, know what is happening on it, and remain in a position to make decisions about it.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Stopping Recursive Self-Improvement Is the Beginning of AI Control by Ezra Klein|Stopping Recursive Self-Improvement Is the Beginning of AI Control by Ezra Klein]] > “Many people at these labs began working on AI because they wanted to better humanity. They began working on AI because they feared incomprehensible, autonomous AI slipping out of humanity's control. They saw what was coming, and they were so right about it that they built some of the most valuable companies with some of the most transformational technology in human history. Now they find themselves racing one another to build incomprehensible, autonomous AIs that they admit are slipping out of humanity's control and beyond even our ability to monitor. This is the tragedy of their work: in fighting against a prophecy, they have drawn it tighter around their necks and ours. It is time to make them stop.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Labs Tightened the Prophecy They Tried to Escape by Ezra Klein|AI Labs Tightened the Prophecy They Tried to Escape by Ezra Klein]]  > “The use of coding agents by OpenAI research teams changed night and day. Those teams were using more than one hundred times as much agentic compute as they had at the beginning of the year.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Agentic AI/OpenAI's Use of Coding Agents Increased One Hundredfold by David Robinson|OpenAI's Use of Coding Agents Increased One Hundredfold by David Robinson]] > “The frontier laboratories say artificial intelligence may be moving too fast and disclose rogue-agent incidents, yet they are devoting enormous internal resources to getting AI systems they do not control to build AI systems they will understand even less, even faster.” > **— Ezra Klein**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Uncontrolled AI Is Building Less Understandable AI Faster by Ezra Klein|Uncontrolled AI Is Building Less Understandable AI Faster by Ezra Klein]] > “I looked around internally and thought, ‘This is nuts. What is happening? This is not right.’ Disclosure only gets us so far when a company says that it is pursuing recursive self-improvement and is not sure that doing so is a good idea.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Disclosure Cannot Excuse Unsafe Recursive Self-Improvement by David Robinson|Disclosure Cannot Excuse Unsafe Recursive Self-Improvement by David Robinson]] > “There is a lot of cognitive dissonance involved in being part of an organization simultaneously warning about recursive self-improvement and pursuing it, particularly as recursive self-improvement becomes real.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/AI Labs Warn About Recursive Self-Improvement While Pursuing It by David Robinson|AI Labs Warn About Recursive Self-Improvement While Pursuing It by David Robinson]] > “My main worry about recursive self-improvement is not velocity. My main worry is that an AI can make a smarter AI in a way human beings are not going to understand.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Recursive AI Could Build Smarter AI Beyond Human Understanding by David Robinson|Recursive AI Could Build Smarter AI Beyond Human Understanding by David Robinson]] > “On the recursive self-improvement path, human beings will increasingly have to defer to models to explain how the models work and whether they are aligned, at a time when we fundamentally have not established that they are aligned.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Recursive AI Will Force Humans to Trust Models to Explain Models by David Robinson|Recursive AI Will Force Humans to Trust Models to Explain Models by David Robinson]] > “Even if frontier laboratories adopted every proven control technique from nuclear power and aviation, we still would not know how to deeply align AI systems and ensure that they do what human beings would reasonably want when nobody is looking. The premise of recursive self-improvement is that human beings will not be looking.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Recursive Self-Improvement Assumes Humans Will Not Be Looking by David Robinson|Recursive Self-Improvement Assumes Humans Will Not Be Looking by David Robinson]] > “I agree that recursive self-improvement should be prohibited until we are sure it is safe, with exceptions carved out only as particular uses are shown to be safe.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Recursive Self-Improvement Should Be Prohibited Until Proven Safe by David Robinson|Recursive Self-Improvement Should Be Prohibited Until Proven Safe by David Robinson]]  > Recursive self-improvement may become the highest-value use of a frontier laboratory’s own tokens. If stronger models generate more future value than outside applications, socially valuable work can lose compute to the development of the next model generation. > **— Adapted from Alexander Wissner-Gross**, *Moonshots, October 7, 2026* [[reminders/Machine Succession/Recursive Self-Improvement May Become the Most Valuable Use of Frontier Tokens by Alexander Wissner-Gross|Recursive Self-Improvement May Become the Most Valuable Use of Frontier Tokens by Alexander Wissner-Gross]] > Recursive self-improvement is constrained by more than compute. Memory, energy, evaluation, and implementation still slow each iteration; the singular transition arrives when AI compresses the innovation cycle faster than institutions and people can follow it. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* [[reminders/Machine Succession/Recursive Self-Improvement Is Constrained by More Than Compute by Salim Ismail|Recursive Self-Improvement Is Constrained by More Than Compute by Salim Ismail]]