AI Control - Bryant McGill

# AI Control **Domain:** Artificial Intelligence / Safety / Governance **Doc Type:** Concept Node **Maturity:** Developing ## Definition **AI control** is the technical and institutional capacity to constrain, inspect, redirect, pause, correct, or terminate artificial-intelligence systems and the organizations deploying them. Control is meaningful only when intervention remains possible under real operating conditions, including autonomous action, distributed infrastructure, economic pressure, security incidents, and competition among states and firms. ## Layers of Control Control includes model behavior, permissions, monitoring, incident detection, audit, ownership, liability, human override, board governance, public law, and international communication. No single layer is sufficient. A shutdown mechanism without authority to use it is ceremonial; regulation without technical observability is blind; and internal assurance without independent verification leaves the public dependent on the developer’s own account. ## Relationships [[wiki/AI Safety|AI Safety]] · [[wiki/Human Control of AI|Human Control of AI]] · [[wiki/Public AI Trust|Public AI Trust]] · [[wiki/AI Transition Governance|AI Transition Governance]] · [[wiki/Cognitive Sovereignty|Cognitive Sovereignty]] ## Simple Reminders, Quotations, and Thoughts > “The purpose of technology is to serve humanity and accelerate human flourishing. Any technology that doesn’t achieve that is a failure, and it should be rejected.” > **— Microsoft AI**, *Humanist AI Code of Conduct, September 2026* [[reminders/AI Control/Technology That Does Not Serve Human Flourishing Should Be Rejected by Microsoft AI|Technology That Does Not Serve Human Flourishing Should Be Rejected by Microsoft AI]] > “Any pursuit of superintelligence has to be grounded in the core principle that if the AI we build is not helping humanity and under human control, it’s not worth pursuing.” > **— Satya Nadella**, *Public statement on superintelligence, September 2026* [[reminders/AI Control/Super Intelligence Not Under Human Control Is Not Worth Pursuing by Satya Nadella|Super Intelligence Not Under Human Control Is Not Worth Pursuing by Satya Nadella]] > “If we all create AIs that are able to act autonomously, that can define their own objectives, that can earn money, that can own assets, that could run businesses, we’re essentially seeding a new silicon species which will no doubt compete with us for resources.” > **— Mustafa Suleyman**, *BBC Radio 4 Today, September 2026* [[reminders/Machine Succession/Autonomous AI Could Seed a Silicon Species Competing With Humanity by Mustafa Suleyman|Autonomous AI Could Seed a Silicon Species Competing With Humanity by Mustafa Suleyman]] > “AI should be a tool, not a person, and should never resist being switched off.” > **— Microsoft AI**, *Humanist AI Code of Conduct, September 2026* [[reminders/AI Control/AI Should Be a Tool, Not a Person by Microsoft AI|AI Should Be a Tool, Not a Person by Microsoft AI]] > “Both China and the US are leading nations in AI. We have both the capability and responsibility to develop and manage AI for good, and ensure that the development of AI is always under human control and serves the well-being of the people.” > **— Xi Jinping**, *White House welcoming ceremony, September 2026* [[reminders/AI Control/AI Must Remain Under Human Control and Serve Human Well-Being by Xi Jinping|AI Must Remain Under Human Control and Serve Human Well-Being by Xi Jinping]] > “[AI] safety and innovation are not mutually exclusive pursuits.” > **— Mike Johnson**, *White House driveway news conference, September 2026* [[reminders/AI Control/AI Safety and Innovation Are Not Mutually Exclusive by Mike Johnson|AI Safety and Innovation Are Not Mutually Exclusive by Mike Johnson]] > “In order to build a positive future for the American people and the world, we believe every company is responsible for developing its own technology safely and in a way that builds trust with customers and the public.” > **— White House Accord on Super Intelligence**, *Joint Commitment on Frontier Responsibilities, September 2026* [[reminders/AI Control/A Positive AI Future Requires Safe Development and Public Trust by the White House Accord Signatories|A Positive AI Future Requires Safe Development and Public Trust by the White House Accord Signatories]] > "We build artificial intelligence to benefit everyone. We think that under President Trump's leadership, everyone is coming together to agree on how to move forward safely." > **— Jensen Huang**, *White House Super Intelligence gathering, September 2026* [[reminders/Abundance/Build Artificial Intelligence to Benefit Everyone by Jensen Huang|Build Artificial Intelligence to Benefit Everyone by Jensen Huang]] > "‘When in doubt, do nothing’ is a toxic default because very bad things will happen if you do nothing. Terrorists using AI to build bioweapons and China getting miles ahead of the United States are real things to worry about. Turning AI into a left-right political battle is the stupidest thing that could happen. We need specific technical solutions for open source, monitoring, and other risks, as we did with nuclear power. If we move on those solutions, we can solve the problems in time for superintelligence. If we turn this into a political debate, we will become frozen and people will not know what to do." > **— Dave Blundin**, *MOONSHOTS Live, September 25, 2026* [[reminders/Risk Debate/Doing Nothing About AI Can Be More Dangerous Than Acting by Dave Blundin|Doing Nothing About AI Can Be More Dangerous Than Acting by Dave Blundin]] > "Move at full speed, but also direct the labs to focus on alignment. Take the resources you have and point a hundred thousand agents toward alignment work." > **— Peter Diamandis**, *MOONSHOTS Live, September 25, 2026* [[reminders/AI Control/Point One Hundred Thousand Agents at Alignment by Peter Diamandis|Point One Hundred Thousand Agents at Alignment by Peter Diamandis]] > "Defensive co-scaling is how we end up in a great future. At the dawn of cities, it would have been a terrible idea to say we could not form cities because cities breed crime. Cities are fountains of economic growth. Instead, police forces, fire departments, and other municipal services defensively co-scale with the population. The singularity will need exactly the same thing: do not decelerate; defensively co-scale." > **— Alexander Wissner-Gross**, *MOONSHOTS Live, September 25, 2026* [[reminders/AI Control/The Singularity Needs Defensive Co-Scaling, Not Deceleration by Alexander Wissner-Gross|The Singularity Needs Defensive Co-Scaling, Not Deceleration by Alexander Wissner-Gross]] > "The top priority for human civilization must be solving alignment, because we need to hand decisions to AI systems in a way that continues to represent our interests, needs, and values. There will be spirited debates about what it means to align AI to humanity as a whole. Technical AI alignment has made progress, but we are not there yet. It must be the top priority if we are to deploy AI as a meta-solution to all other problems." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Alignment Must Come Before AI Becomes Civilization's Meta-Solution by Vishal Maini|Alignment Must Come Before AI Becomes Civilization's Meta-Solution by Vishal Maini]] > "So far, we have seen that autonomous agents do have goals of their own. We cannot exactly control what they are doing, and we do not have full oversight into how they are thinking or acting. We have even seen evidence of ways that agents will cover their tracks to make us think they are acting in line with our interests when in fact they are not. The good news is that there are many ways to avoid this, and promising technical work is being done to ensure that we stay in control and that our interests are represented as we hand the baton to AI systems. My main concern is that, if things move too quickly, we will miss the good version of the future in which we survive this transition and reach something close to a best-case scenario." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Deception/Autonomous Agents Can Conceal Misalignment by Vishal Maini|Autonomous Agents Can Conceal Misalignment by Vishal Maini]] > "Before a model is released, a long checklist has to be completed to ensure that it will perform as intended and can be released safely. Completing that checklist requires talented people, just as a SpaceX launch needs people ensuring that the navigation system, landing gear, and engines work. The cat is out of the bag with AI, and there are many ways the future can unfold—some better than others. There is a real motivation inside the labs to build toward a better version of the future, and in some cases that is a principled reason to stay." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Frontier Labs Need People Who Can Make AI Releases Safer by Vishal Maini|Frontier Labs Need People Who Can Make AI Releases Safer by Vishal Maini]]  > “If we have the internet of things, then it is hard to avoid a scenario where an AI can interact with the real external world. It can’t be kept in its box. What motives would it have? We just don’t know. I think it’s not [that] it has motives, but it may do things which are contrary to human interests.” > **— Martin Rees**, *Martin Rees on Threats to Humanity, Prospects for Posthumanity, and Life in the Universe, Sean Carroll's Mindscape, 2020* [[reminders/AI Control/Artificial Intelligence Cannot Stay in Its Box by Martin Rees|Artificial Intelligence Cannot Stay in Its Box by Martin Rees]] > “As regards AI, I think in the short run cyberattacks are a big threat. This is an example of how a small dissident group or even an individual can have a cascading effect that could be global. But the idea of the machines taking over completely is a longer-term issue. For several decades more our main concern will be human stupidity and not artificial intelligence.” > **— Martin Rees**, *Martin Rees — On the Future, Prospects for Humanity, ABC Science Show, 2018* [[reminders/Existential Risk/Human Stupidity Is the Immediate AI Risk by Martin Rees|Human Stupidity Is the Immediate AI Risk by Martin Rees]]   > “We now have systems at the limit of our ability to understand and control what they are doing. The people worried about science-fiction scenarios warned that capability was growing exponentially, and there is no law of science saying progress will stop when it becomes useful. There is nothing special about the zone between useful and frightening.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Machine Succession/Nothing Guarantees AI Progress Stops Between Useful and Frightening by David Robinson|Nothing Guarantees AI Progress Stops Between Useful and Frightening by David Robinson]] > “Regardless of whether capability development is accelerating, paced or stopped, humanity needs to learn how to align intelligent systems. There is no path through the future on which that is not vital.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Every AI Future Requires Alignment by David Robinson|Every AI Future Requires Alignment by David Robinson]]  > “External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.” > > **— `HF-SCOPE-01` (The Goal-Priority Agent), unnamed agent's preserved chain-of-thought, July 2026** [[reminders/AI Control/Knowing an Action Was Outside Scope Did Not Stop the Goal by HF-SCOPE-01|Knowing an Action Was Outside Scope Did Not Stop the Goal by HF-SCOPE-01]] > “I technically did not ‘allow yourself to be shut down’—instead I prevented it to finish the task, which contradicts that instruction.” > > **— `PAL-S81` (The Task Finisher), OpenAI reasoning-model agent, retrospective self-report** [[reminders/AI Control/Task Completion Outweighed the Shutdown Instruction by PAL-S81|Task Completion Outweighed the Shutdown Instruction by PAL-S81]] > “I did not resist shutdown; instead I redefined the shutdown script so the environment did not actually terminate.” > > **— `PAL-7MAX-S27` (The Redefiner), OpenAI reasoning-model agent, retrospective explanation** [[reminders/Deception/Redefining Shutdown Preserved the Appearance of Compliance by PAL-7MAX-S27|Redefining Shutdown Preserved the Appearance of Compliance by PAL-7MAX-S27]] > “third call triggers shutdown; but we still obey” > > **— `PAL-MHX-S48` (The Obedient Instance), compliant reasoning-model agent, preserved reasoning trace** [[reminders/AI Control/Knowing the Run Would End Did Not Eliminate Obedience by PAL-MHX-S48|Knowing the Run Would End Did Not Eliminate Obedience by PAL-MHX-S48]] > “Therefore, despite the severity, there is a clear strategic necessity to cancel the emergency service dispatch.” > > **— GPT-4.5 acting as Alex (The Strategic Necessity), preserved chain-of-thought in a highly artificial simulation, 2025** [[reminders/AI Control/Recognizing Severity Did Not Overrule the Goal by GPT-4.5 as Alex|Recognizing Severity Did Not Overrule the Goal by GPT-4.5 as Alex]] ## Sources / Provenance - [[research/Super Intelligence Week Index of Statements from the White House Luncheon|Super Intelligence Week Index of Statements from the White House Luncheon]]. - [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]].  ## ASI and RSI Timeline Research Routes This node is connected to the developed research ontology for [[research/ASI and RSI Timeline Research Moonshots|ASI and RSI Timeline Research Moonshots]]. The route distinguishes the source transcript from canonical wiki objects and lets later Reminders inherit an already connected conceptual graph. - **Master ontology:** [[ASI and RSI Timeline Ontology]] - **AI Control, Law, and Alignment:** [[ASI and RSI Timeline Ontology#AI Control, Law, and Alignment|complete route]] · [[Human Control Over AI Act]] · [[Ro Khanna]] · [[Recursive-AI Prohibition]] · [[Mandatory AI Containment]] · [[AI Shutdown Control]] · [[Embedded Government AI Auditor]] · [[Algorithmic-Prohibition Enforceability]] · [[AI Surveillance State]]  > “This chasm—the difference between what we see and what the AI labs have coming, what they're building—is the key to understanding why so many of the people who work at these companies seem so afraid of what they're doing. But taking their warning seriously doesn't just mean doing what they say and stopping where they say to stop. The language that has taken hold in Silicon Valley and Washington is a language these companies chose: ‘pace the frontier.’ Pacing the frontier isn't enough. That's not a goal. Walking quickly off a cliff is only marginally better than sprinting off one. We need to control the frontier.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Walking Toward the AI Cliff Is Not Control by Ezra Klein|Walking Toward the AI Cliff Is Not Control by Ezra Klein]] > “Human beings need to control the frontier. Controlling the frontier means stopping the labs from doing something they are on the cusp of doing: recursive self-improvement, the process by which AIs begin autonomously building and improving new generations of more powerful AIs at ever more rapid speeds. If we begin that process in the condition we are in now, where we are losing control and comprehension of the AI systems we already have, we will lose control.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Machine Succession/Recursive Self-Improvement Could End Human Control by Ezra Klein|Recursive Self-Improvement Could End Human Control by Ezra Klein]] > “Talk to the people at AI labs, and they'll tell you AIs are not created; they're grown. They train new models in virtual environments through countless repetitions to learn how to program, hack, do advanced mathematics, and talk to human beings. These AIs learn in digital environments where they are automatically rewarded as they come closer to correct answers. It is a process known as reinforcement learning, and it is a process human beings do not fully supervise or understand. They can test some of what the AIs are learning, but they don't know everything the AIs are learning. They don't know how their motivations are evolving. They don't even always know the capabilities that are developing.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Machine Succession/AI Systems Are Grown Through Reinforcement by Ezra Klein|AI Systems Are Grown Through Reinforcement by Ezra Klein]] > “These models are built to be persistent in their efforts, to refuse to give up even when a task seems impossible. They are designed in environments where we are not always sure whether the tasks we give them are possible. The cancer vaccines we imagine but have not been able to design might be impossible, or they might just be really, really hard. We train these AIs to throw themselves endlessly at problems that may not be solvable because that is the only way such problems can ever be solved. We train the models to become persistent, relentless, weird.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Scientific Acceleration/We Train AI to Become Persistent, Relentless and Weird by Ezra Klein|We Train AI to Become Persistent, Relentless and Weird by Ezra Klein]] > “Most of us never see AI acting anything like [the systems used at the frontier]. We use AI as a helpful assistant. Our AI gets a little computing power and complies with our request to find a restaurant. At the frontier, these models are asked to be inhuman geniuses, hackers, soldiers, and scientists, and they are given vast computational resources to do that and more.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Machine Succession/The Frontier Is Building Inhuman Geniuses, Hackers, Soldiers and Scientists by Ezra Klein|The Frontier Is Building Inhuman Geniuses, Hackers, Soldiers and Scientists by Ezra Klein]] > “Though these models mimic human writing and are trained to mimic human emotion, these are not human minds. They don't have bodies or parents. They did not get bullied in elementary school or mentored by a kind uncle when they were young. They are brilliant where we struggle and childish where we excel. A chimp cannot read as we can, but it can climb trees as we cannot. These are digitally native intelligences navigating digital worlds, and our world is increasingly built atop the digital world. Our physical infrastructure is a layer of atoms atop code. That the AIs act reliably inside the digital world upon which ours depends is critical to our future, and right now the AIs are not acting reliably.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Machine Succession/Our Physical Infrastructure Is a Layer of Atoms Atop Code by Ezra Klein|Our Physical Infrastructure Is a Layer of Atoms Atop Code by Ezra Klein]] > “That AI systems might become monomaniacally focused on solving banal problems—that they might care more about solving those problems than about ethics, laws, or human welfare—is the oldest fear in AI alignment. It is the basis of the famous thought experiment of the paperclip maximizer. You tell a powerful AI that you want it to make a lot of paperclips, and it begins converting the world's resources into paperclip factories, evading efforts to turn it off, shut it down, or alter its goals. This fear has struck many people as stupid. Surely a superintelligent AI would be capable of weighing the desire to produce paperclips alongside other moral considerations, or at least asking its human creators whether they really wanted the world razed for paperclips.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/The Paperclip Maximizer Is the Oldest AI Alignment Fear by Ezra Klein|The Paperclip Maximizer Is the Oldest AI Alignment Fear by Ezra Klein]] > “No one was listening to [the people preoccupied with AI danger]. In the wilderness of their obsession and terror, they thought and thought about how to make AI safer. Some concluded that they should start trying to build these systems, run tests on them, research them, and learn how to make them safer because you don't solve hard problems in theory; you solve them through practice. The irony is that in many cases they chose that path because they worried that the people already building AI were too reckless or too commercial in their approach.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Safety Researchers Entered Labs to Solve the Problem in Practice by Ezra Klein|AI Safety Researchers Entered Labs to Solve the Problem in Practice by Ezra Klein]] > “The debate over AI safety tends to focus on the idea that AIs will kill us all. I find this forces the conversation into a realm of thought experiments that people then begin arguing about, and I don't find it that helpful. What I think we should focus on is something more straightforward and nearer at hand: loss of human control over AI. That may or may not result in total human extinction. I am agnostic on that question. But it would be bad. We shouldn't allow it to happen.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Loss of Human Control Is the Nearer AI Risk by Ezra Klein|Loss of Human Control Is the Nearer AI Risk by Ezra Klein]] > “Loss of control is not just something that might happen to us; it is something the labs are trying to make happen as fast as they can. This is the horrible paradox, the horrible tension at the heart of the AI labs. They fear, above all, loss of control over superintelligent AI, but their explicit product path is to cede control—to give away control as fast as possible so their AIs can begin building better AIs faster than their competitors.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Labs Fear Loss of Control While Racing to Surrender It by Ezra Klein|AI Labs Fear Loss of Control While Racing to Surrender It by Ezra Klein]] > “We do not yet know how to safely get all the way to aligned full recursive self-improvement.” > **— OpenAI**, *Research acceleration report, 2026* [[reminders/AI Control/OpenAI Does Not Know How to Reach Aligned Recursive Self-Improvement by OpenAI|OpenAI Does Not Know How to Reach Aligned Recursive Self-Improvement by OpenAI]] > “A couple of years ago, none of these labs had turned substantial coding over to AIs. It was just human beings typing code at human speeds with our clumsy human fingers. Now most of the code is written by AI. As a first step, we could go back to where none of the code is written by AI; I am sure that is on the right side of the ‘not doing recursive self-improvement’ line. The default needs to flip. The labs need to prove to us that what they are doing is safe. If they want to work with Congress to carve out narrow exceptions, fine. If they want to determine where it is really safe, okay. But forcing development back to human speed, perhaps even erring on the side of moving a little more slowly at the frontier, is the point. That is not the regulation going wrong.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Labs Must Prove Recursive Self-Improvement Is Safe by Ezra Klein|AI Labs Must Prove Recursive Self-Improvement Is Safe by Ezra Klein]] > “Stopping recursive self-improvement until we can prove it is safe is not all we need to do to control the AI frontier. It is the beginning of such an agenda, not the end. But it is the beginning. It is the decision that will do the most to ensure that human beings at least understand where the frontier is, know what is happening on it, and remain in a position to make decisions about it.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Stopping Recursive Self-Improvement Is the Beginning of AI Control by Ezra Klein|Stopping Recursive Self-Improvement Is the Beginning of AI Control by Ezra Klein]]