AI Governance - Bryant McGill

# AI Governance **Domain:** Artificial Intelligence / Law / Institutions **Doc Type:** Canonical Governance Node **Maturity:** Evolving ## Definition **AI governance** is the set of institutions, laws, technical controls and participatory processes that shape how machine-intelligence systems are built, evaluated, deployed and contested. It is distinct from [[wiki/Algorithmic Governance|Algorithmic Governance]], which addresses governing people and resources through models, and from [[wiki/Computational Governance|Computational Governance]], the broader use of computational systems in collective administration. The domains overlap whenever an AI system becomes part of a consequential governing workflow. ## Evolutionary Nexus [[articles/The Evolutionary Roots of Silicon Valley|The Evolutionary Roots of Silicon Valley]] supplies two precedents: Asilomar's explicit risk constraints and the public Human Genome Project's open infrastructure. One governs hazardous experimentation; the other governs access to foundational knowledge. ## Continuity Context Governance must cover systems as actors and as substrates for human continuity. It therefore includes model accountability, energy and custody, portability, identity evidence, appeal and the rights of possible machine subjects. ## Key Insight **AI governance must constrain power without freezing the ontology of who may eventually possess standing.** ## See Also [[wiki/Algorithmic Governance|Algorithmic Governance]], [[wiki/Constitutional Governance|Constitutional Governance]], [[wiki/Precautionary Principle|Precautionary Principle]], [[wiki/Moral Accountability Layer|Moral Accountability Layer]] ## Simple Reminders, Quotations, and Thoughts > "A.I. rights are liable to expand to more and more A.I. over time." > **— Moshe Hoffman**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/AI Control/Artificial Intelligence Rights Will Expand by Moshe Hoffman|Artificial Intelligence Rights Will Expand by Moshe Hoffman]] > "A human-made information processor could, in principle, duplicate and exceed the powers of the human mind." > **— Steven Pinker**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/AI Control/Information Processors Could Exceed the Human Mind by Steven Pinker|Information Processors Could Exceed the Human Mind by Steven Pinker]] > "To keep clean our consciences, we need only to create a thinking machine, and then vilify it." > **— Kurt Gray**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Machine Succession/Thinking Machines Can Launder Human Conscience by Kurt Gray|Thinking Machines Can Launder Human Conscience by Kurt Gray]] > "A machine that thinks won't always think in the ways we want it to." > **— Bruce Schneier**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/AI Control/Thinking Machines Will Not Always Obey by Bruce Schneier|Thinking Machines Will Not Always Obey by Bruce Schneier]] > "Typicality implies our likely demise in the next million years. But it tells us nothing about whether this will come at the hands (or other appendages) of an artificial intelligence; after all, there is no shortage of doomsday scenarios." > **— Raphael Bousso**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Existential Risk/Typicality Predicts Humanity’s Demise Within a Million Years by Raphael Bousso|Typicality Predicts Humanity’s Demise Within a Million Years by Raphael Bousso]] > “Artificial intelligence is an intention multiplier. It does not give a state, corporation, political party, military organization, religion, or civilization a purpose of its own; it multiplies the capacity with which an existing purpose can be pursued through research, planning, communication, optimization, surveillance, administration, defense, finance, and execution.” > **— Bryant McGill**, *The Political Party That Inherits Super Intelligence in 2028, 2026* [[reminders/AI Control/AI Multiplies the Intentions It Inherits by Bryant McGill|AI Multiplies the Intentions It Inherits by Bryant McGill]] > "Having a technical expert in the room as legislation is drafted makes it easier to align standards and policies with the technical reality of a fast-moving frontier. There are many ways for policy to poison the well or backfire when it is not grounded in technical reality. Policymakers should be in direct conversation with the people developing these models." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/AI Policy Must Remain Grounded in Technical Reality by Vishal Maini|AI Policy Must Remain Grounded in Technical Reality by Vishal Maini]] > "When I joined DeepMind, only a small number of people were working on AI safety, and little dedicated work was happening on long-term strategy and governance. I spent four years helping to get those efforts off the ground. My departure was more a personal decision than a comment on where the lab was going. I am proud of the progress the company has made, and I believe some of that early work made it more possible to advance safety and governance—work of which we still need much more." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/AI Safety and Governance Still Need Much More Work by Vishal Maini|AI Safety and Governance Still Need Much More Work by Vishal Maini]] > If you truly believe we are not far from human-level AI and that its risks are urgent, you make different prioritization decisions. The difference is not that less-convinced people do not care about making AI controllable or safe; it is that the problem feels far away. Immediacy determines whether someone will spend extra energy, political capital, research effort, compute, or reputational risk on a topic that otherwise seems frightening, speculative, and distant. > **— Adapted from Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Belief in Near-Term AGI Changes What People Are Willing to Risk by Vishal Maini|Belief in Near-Term AGI Changes What People Are Willing to Risk by Vishal Maini]] > "Human survival is not a partisan issue. This is one of those rare opportunities in which literally every human being has a common cause, and it deserves to be treated that way." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Existential Risk/Human Survival Is Not a Partisan Issue by Vishal Maini|Human Survival Is Not a Partisan Issue by Vishal Maini]] > "When deciding which AI leaders to trust, it is worth looking at their track records before they had financial incentives or platforms to protect. Reading what industry leaders said more than a decade ago can help cut through claims that their current positions are merely regulatory capture." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Trust AI Leaders by What They Said Before Power and Money Arrived by Vishal Maini|Trust AI Leaders by What They Said Before Power and Money Arrived by Vishal Maini]]  > “OpenAI was a very decentralized place. People would debate what to do and finally say, ‘We aligned that this is the right next thing,’ which functionally meant that they agreed. The unanswered question of who had the decision rights was rendered moot, so no one had to figure it out.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Decentralization Left OpenAI Decision Rights Unanswered by David Robinson|Decentralization Left OpenAI Decision Rights Unanswered by David Robinson]] > “Part of the fundamental cognitive dissonance for me was that we kept publishing warnings while ultimately continuing to train and deploy the dangerous models we were warning about. No matter how many warnings we publish, we have to ask whether what we are doing is actually reasonable.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/AI Labs Warn About the Models They Continue to Deploy by David Robinson|AI Labs Warn About the Models They Continue to Deploy by David Robinson]] > “I looked at the people and environment around me and asked what my loved ones—or strangers—would want OpenAI to be doing if the experts warning about a meaningful chance of catastrophic and irreversible loss of control were right. I do not know whether they are right, but the caution people would reasonably expect frontier laboratories to exercise is totally unlike anything I have heard of happening in the industry.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Frontier AI Labs Exercise Too Little Caution by David Robinson|Frontier AI Labs Exercise Too Little Caution by David Robinson]] > “The question is not whether the organization has brakes; things have been stopped. The question is how willing it is to stop when push comes to shove. Public descriptions of what has been paused may be true while remaining very carefully scoped.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/The Test of AI Brakes Is Willingness to Stop by David Robinson|The Test of AI Brakes Is Willingness to Stop by David Robinson]] > “We need to meet safety criteria. If we can meet them in ten minutes, great. If we stop for two months and still have not met them, then we are still blocked. Running off a cliff and walking slowly off a cliff are not that different.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Slowing Down Does Not Satisfy Unmet AI Safety Criteria by David Robinson|Slowing Down Does Not Satisfy Unmet AI Safety Criteria by David Robinson]] > “I worry that we do not have a coherent concept beneath the word ‘aligned.’ OpenAI’s accounts of its work often relied upon huge abstractions such as ‘give society time to get ready’ or ‘benefit all of humanity,’ but society is not one thing and human values are not one set of values. They are a cacophony—beautiful, but messy.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Civilization/Human Values Are a Beautiful Cacophony by David Robinson|Human Values Are a Beautiful Cacophony by David Robinson]] > “Democratizing useful AI tools is fine as far as it goes, but how far does it go? We are now discussing a level of risk that, if the concern is correct, nobody wants to be taking.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Risk Debate/Democratizing AI Does Not Democratize Unwanted Risk by David Robinson|Democratizing AI Does Not Democratize Unwanted Risk by David Robinson]] > “I agree that recursive self-improvement should be prohibited until we are sure it is safe, with exceptions carved out only as particular uses are shown to be safe.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Recursive Self-Improvement Should Be Prohibited Until Proven Safe by David Robinson|Recursive Self-Improvement Should Be Prohibited Until Proven Safe by David Robinson]] > “Changes in what is technologically possible can rapidly change what seems politically plausible. When I joined OpenAI, the Overton window felt like an Overton door into a strange world where reasonable possibilities were entirely outside what I had previously considered normal. That kind of change could also occur in United States–China cooperation on artificial intelligence.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Superpower/Technological Change Can Open the Door to US-China AI Cooperation by David Robinson|Technological Change Can Open the Door to US-China AI Cooperation by David Robinson]] > “Artificial intelligence is not precisely like anything humanity has governed before, but that does not mean we face a blank page. Nuclear power is an analogy, aviation is an analogy, and dealing with people and organizations is an analogy. Human beings have valuable tools for figuring out unfamiliar problems.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/AI Governance Does Not Begin from a Blank Page by David Robinson|AI Governance Does Not Begin from a Blank Page by David Robinson]] > “Whether human beings should create superintelligence depends on what superintelligence becomes, and we do not know what it will become. From the beginning of my time at OpenAI, I would ask what a good future looked like and where we were trying to go. People who were otherwise proud and confident would say that was above their pay grade, that people would figure it out or that the future should become whatever people wanted. The sense of the good beneath the project felt impoverished.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Civilization/Superintelligence Is Being Built Without a Vision of the Good by David Robinson|Superintelligence Is Being Built Without a Vision of the Good by David Robinson]] > “We have to make good choices. We have no time to rush.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Civilization/We Have No Time to Rush by David Robinson|We Have No Time to Rush by David Robinson]]  > A classic twentieth-century task force is structurally mismatched to a distributed twenty-first-century technology. Administrative centrality is not the same thing as technical control. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* [[reminders/AI Control/Centralized Task Forces Cannot Control Distributed AI by Salim Ismail|Centralized Task Forces Cannot Control Distributed AI by Salim Ismail]] > Distributed AI needs scaffolding rather than imagined central control: incident reporting, identity, liability, bounded experiments, instrumentation, published failures, and accountable responses. Governance should reduce the blast radius without pretending it can centrally command an intelligence explosion. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* [[reminders/AI Control/Distributed AI Needs Scaffolding Around Its Blast Radius by Salim Ismail|Distributed AI Needs Scaffolding Around Its Blast Radius by Salim Ismail]] > Calling superintelligence a race does not explain victory. What threshold ends the race, who declares the winner, and does winning require suppressing everyone else’s capacity? Competition can be real while the metaphor remains strategically incomplete. > **— Adapted from Dave Blundin**, *Moonshots, October 7, 2026* [[reminders/AI Superpower/Calling AI a Race Does Not Explain What Winning Means by Dave Blundin|Calling AI a Race Does Not Explain What Winning Means by Dave Blundin]] > Calling this a “1942 moment” does not establish a historical equivalence with the Manhattan Project. It signals an intention to accelerate, and it tells Congress, industry, and foreign states that the United States means to treat superintelligence as a national competition it intends to win. > **— Adapted from Alexander Wissner-Gross**, *Moonshots, October 7, 2026* [[reminders/AI Superpower/A 1942 Moment Signals Acceleration Not Historical Equivalence by Alexander Wissner-Gross|A 1942 Moment Signals Acceleration Not Historical Equivalence by Alexander Wissner-Gross]] > A bilateral AI hotline may help two powers manage agents running amok without ending their competition. An incident channel can reduce misunderstanding while the underlying superintelligence arms race continues. > **— Adapted from Alexander Wissner-Gross**, *Moonshots, October 7, 2026* [[reminders/AI Superpower/An AI Hotline Can Manage Incidents Without Ending the Arms Race by Alexander Wissner-Gross|An AI Hotline Can Manage Incidents Without Ending the Arms Race by Alexander Wissner-Gross]] > American AI policy is pulling the accelerator and the brake at the same time: one senior official warns that the danger is moving too slowly while another tells frontier laboratories to slow down. A state cannot coordinate acceleration until it resolves that contradiction. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* [[reminders/AI Superpower/American AI Policy Is Pulling the Accelerator and Brake at Once by Salim Ismail|American AI Policy Is Pulling the Accelerator and Brake at Once by Salim Ismail]] > The presence of danger does not settle whether a powerful technology should exist. The governing problem is how to extract the promise without the peril by building guardrails that preserve civilization-scale benefits while confronting the harms honestly. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* [[reminders/Risk Debate/Promise and Peril Are Problems of Guardrail Design by Salim Ismail|Promise and Peril Are Problems of Guardrail Design by Salim Ismail]] > AI risk is not a single knob that society turns left for safety or right for progress. Risk reduction is an engineering problem: identify specific failure modes, design specific mitigations, and preserve the benefits that blanket suppression would destroy. > **— Adapted from Dave Blundin**, *Moonshots, October 7, 2026* [[reminders/Risk Debate/AI Risk Is Not a Single Knob by Dave Blundin|AI Risk Is Not a Single Knob by Dave Blundin]] > Once capable model weights are widely available, governance shifts from controlling the core technology toward regulating harmful outcomes, guardrails, and the on-ramps and off-ramps through which systems reach the world. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* [[reminders/AI Control/Open Weights Move Governance Toward Outcomes and Access Points by Salim Ismail|Open Weights Move Governance Toward Outcomes and Access Points by Salim Ismail]] > AI governance needs a map of what could go right and what could go wrong, stated by the people making each case. Expected utility helps weigh probable outcomes; minimax regret identifies losses society cannot accept. Policy should respond to the mapped gaps instead of governing from undifferentiated fear. > **— Adapted from Emad Mostaque**, *Moonshots, October 7, 2026* [[reminders/Risk Debate/AI Governance Should Map Expected Benefit and Catastrophic Regret by Emad Mostaque|AI Governance Should Map Expected Benefit and Catastrophic Regret by Emad Mostaque]] > Willingness to pay is not the same thing as public value. A persuasive consumer system can absorb scarce GPUs because selling products is lucrative while disease research goes unfunded, which is why governments and universities must participate in deciding who gets compute and why. > **— Adapted from Dave Blundin**, *Moonshots, October 7, 2026* [[reminders/AI Infrastructure/Market Allocation Can Send Compute to Persuasion Instead of Public Health by Dave Blundin|Market Allocation Can Send Compute to Persuasion Instead of Public Health by Dave Blundin]] > Open-model capability can exist without being released when downstream liability is broad, ambiguous, and larger than the expected upside. Liability rules do not merely punish harmful deployment after the fact; they can determine whether a market appears at all. > **— Adapted from Dave Blundin**, *Moonshots, October 7, 2026* [[reminders/AI Control/Liability Can Suppress Open-Model Releases by Dave Blundin|Liability Can Suppress Open-Model Releases by Dave Blundin]] ## Relationships - **source argument:** [[articles/The Political Party That Inherits Super Intelligence in 2028|The Political Party That Inherits Super Intelligence in 2028]]. - **conceptual bridge:** [[wiki/Intention Multiplier|Intention Multiplier]]. > “This chasm—the difference between what we see and what the AI labs have coming, what they're building—is the key to understanding why so many of the people who work at these companies seem so afraid of what they're doing. But taking their warning seriously doesn't just mean doing what they say and stopping where they say to stop. The language that has taken hold in Silicon Valley and Washington is a language these companies chose: ‘pace the frontier.’ Pacing the frontier isn't enough. That's not a goal. Walking quickly off a cliff is only marginally better than sprinting off one. We need to control the frontier.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Walking Toward the AI Cliff Is Not Control by Ezra Klein|Walking Toward the AI Cliff Is Not Control by Ezra Klein]] > “With AI advancing at a staggering speed, we must ensure its development is for the positive, for good and for humanity. We must make its oversight and governance precise and effective, and constantly refine measures to forestall loss of control.” > **— Xi Jinping**, *World AI Conference address, 2026* [[reminders/Digital Government/AI Governance Must Forestall Loss of Control by Xi Jinping|AI Governance Must Forestall Loss of Control by Xi Jinping]] > “Anthropic has been relentlessly calling for regulation to slow the pace of development. Regulation would arguably harm Anthropic the most, because it has often been the company furthest out on the AI frontier, and recursive self-improvement is a process by which it could race forward even faster.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Anthropic Calls for Regulation That Could Slow Its Own Lead by Ezra Klein|Anthropic Calls for Regulation That Could Slow Its Own Lead by Ezra Klein]] > “I think it's very hard to say what a ban on recursive self-improvement means. I also think it probably wouldn't be enough.” > **— Sam Altman**, *Fortune interview, September 2026* [[reminders/AI Control/A Ban on Recursive Self-Improvement May Not Be Enough by Sam Altman|A Ban on Recursive Self-Improvement May Not Be Enough by Sam Altman]] > “Where these labs are located, you cannot build an eight-story apartment building without an agonizing public-review process—and probably not even then. Yet somehow these labs can unleash a swarm of 40,000 AI agents to build a society-altering superintelligence without so much as a hearing. OpenAI would need permits to cover its parking lot in solar panels, but it can accelerate into recursive self-improvement whenever it chooses. There is nothing inevitable about any of that. These are political choices, and we can and should make other ones.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Digital Government/AI Labs Face More Review for Solar Panels Than Superintelligence by Ezra Klein|AI Labs Face More Review for Solar Panels Than Superintelligence by Ezra Klein]] > “Stopping recursive self-improvement until we can prove it is safe is not all we need to do to control the AI frontier. It is the beginning of such an agenda, not the end. But it is the beginning. It is the decision that will do the most to ensure that human beings at least understand where the frontier is, know what is happening on it, and remain in a position to make decisions about it.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Stopping Recursive Self-Improvement Is the Beginning of AI Control by Ezra Klein|Stopping Recursive Self-Improvement Is the Beginning of AI Control by Ezra Klein]]