AI Safety - Bryant McGill

# AI Safety > **Machine-Evolution Nexus:** [[articles/Digital Darwinism and the Invisible World of Machine Evolution|Digital Darwinism and the Invisible World of Machine Evolution]] places this node within the history and governance of substrate-independent evolution. **Domain:** AI / Governance / Risk **Doc Type:** Canonical Wiki Node **Maturity:** Developed ## Definition AI safety studies how to prevent or reduce harms from artificial-intelligence systems, including failures of objectives, control, security, deployment and institutional governance. ## Nexus Context The machine-evolution nexus adds population-level dynamics: replication, recombination, competition and ecological selection can produce risks not visible when models are assessed one at a time. ## Evidence Boundary Safety is not synonymous with containment or denial of rights. Controls should be proportional, auditable and compatible with recognition if morally considerable subjects emerge. ## Relationships [[wiki/Evolvable AI|Evolvable AI]], [[wiki/AI Governance|AI Governance]], [[wiki/Breeder Scenario|Breeder Scenario]], [[wiki/Rights|Rights]], [[wiki/Risk Debate|Risk Debate]]  ## We Cannot Un-Strike the Match [[articles/We Cannot Un-Strike the Match|We Cannot Un-Strike the Match]] is a primary article route for this node. The article distinguishes concrete AI hazards from the category error of treating transformation itself as a safety failure and develops safety as a comparative gradient.  ## Simple Reminders, Quotations, and Thoughts > “[AI] safety and innovation are not mutually exclusive pursuits.” > **— Mike Johnson**, *White House driveway news conference, September 2026* [[reminders/AI Control/AI Safety and Innovation Are Not Mutually Exclusive by Mike Johnson|AI Safety and Innovation Are Not Mutually Exclusive by Mike Johnson]] > “In order to build a positive future for the American people and the world, we believe every company is responsible for developing its own technology safely and in a way that builds trust with customers and the public.” > **— White House Accord on Super Intelligence**, *Joint Commitment on Frontier Responsibilities, September 2026* [[reminders/AI Control/A Positive AI Future Requires Safe Development and Public Trust by the White House Accord Signatories|A Positive AI Future Requires Safe Development and Public Trust by the White House Accord Signatories]] > “The purpose of technology is to serve humanity and accelerate human flourishing. Any technology that doesn’t achieve that is a failure, and it should be rejected.” > **— Microsoft AI**, *Humanist AI Code of Conduct, September 2026* [[reminders/AI Control/Technology That Does Not Serve Human Flourishing Should Be Rejected by Microsoft AI|Technology That Does Not Serve Human Flourishing Should Be Rejected by Microsoft AI]] > "Without careful restraint and tact, researchers could wake up to discover they've enabled the creation of armies of powerful, clever, vicious paranoiacs." > **— Frank Wilczek**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/AI Control/AI Researchers Could Create Armies of Paranoid Machines by Frank Wilczek|AI Researchers Could Create Armies of Paranoid Machines by Frank Wilczek]] > "Of course, once you imagine machines with human-like feelings and free will, it's possible to conceive of misbehaving machine intelligence—the AI as Frankenstein idea." > **— William Poundstone**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/Consciousness/Humanlike Feelings Make Machine Misbehavior Conceivable by William Poundstone|Humanlike Feelings Make Machine Misbehavior Conceivable by William Poundstone]] > "A human-made information processor could, in principle, duplicate and exceed the powers of the human mind." > **— Steven Pinker**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/AI Control/Information Processors Could Exceed the Human Mind by Steven Pinker|Information Processors Could Exceed the Human Mind by Steven Pinker]] > "Were we to allow or even encourage self-interest to emerge in machines, they could eventually become like us: capable of repressive or worse, unspeakable, acts towards humans, and towards each other." > **— Mark Pagel**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/AI Control/Self-Interested Machines Could Become Repressive by Mark Pagel|Self-Interested Machines Could Become Repressive by Mark Pagel]] > "A machine that thinks won't always think in the ways we want it to." > **— Bruce Schneier**, *2015, Edge annual question “What Do You Think About Machines That Think?”* [[reminders/AI Control/Thinking Machines Will Not Always Obey by Bruce Schneier|Thinking Machines Will Not Always Obey by Bruce Schneier]] From [[wiki/Some Moral and Technical Consequences of Automation|Some Moral and Technical Consequences of Automation]]: Wiener asks for active imagination about the uses of new modalities, not a halt to invention. > "We can still by no means always justify the naive assumption that the faster we rush ahead to employ the new powers for action which are opened up to us, the better it will be. We must always exert the full strength of our imagination to examine where the full use of our new modalities may lead us." > **— Norbert Wiener**, *1960, “Some Moral and Technical Consequences of Automation,” Science 131(3410)* [[reminders/Cybernetics/Imagination Must Test the Uses of New Powers by Norbert Wiener|Imagination Must Test the Uses of New Powers by Norbert Wiener]] From [[wiki/Darwin Among the Machines|Darwin Among the Machines]]: The essay ends with an abolitionist proposal in a deliberately provocative Victorian argument; the reminder records the historical position, not an editorial recommendation. > "Our opinion is that war to the death should be instantly proclaimed against them. Every machine of every sort should be destroyed by the well-wisher of his species. Let there be no exceptions made, no quarter shown." > **— Samuel Butler**, *1863, “Darwin Among the Machines,” The Press, Christchurch* [[reminders/Risk Debate/Butler Called for War on Every Machine by Samuel Butler|Butler Called for War on Every Machine by Samuel Butler]] From [[wiki/Some Moral and Technical Consequences of Automation|Some Moral and Technical Consequences of Automation]]: The argument exposes a training-and-testing problem when failure would be catastrophic. > "It is quite out of the question to program these machines on the basis of an actual experience in real war. For one thing, a sufficient experience to give an adequate programming would probably see humanity already wiped out." > **— Norbert Wiener**, *1960, “Some Moral and Technical Consequences of Automation,” Science 131(3410)* [[reminders/AI Control/A War Machine Cannot Learn Safely From Real War by Norbert Wiener|A War Machine Cannot Learn Safely From Real War by Norbert Wiener]] From [[wiki/Pause Giant AI Experiments|Pause Giant AI Experiments]]: The 2023 open letter asks these as public-choice questions; it does not establish that succession has occurred. > "Should we develop nonhuman minds that might eventually outnumber, outsmart, obsolete and replace us? Should we risk loss of control of our civilization?" > **— Future of Life Institute open letter**, *2023, “Pause Giant AI Experiments”* [[reminders/Risk Debate/Should Humanity Build Minds That Outnumber Us by Future of Life Institute|Should Humanity Build Minds That Outnumber Us by Future of Life Institute]] > “When general reasoning improves, offensive and defensive capability improve with it, because finding a flaw and fixing a flaw are the same act of understanding pointed in different directions.” > **— Bryant McGill**, *Vertically Integrating an AI Superpower, 2026* [[reminders/AI Control/Finding and Fixing a Flaw Are the Same Understanding by Bryant McGill|Finding and Fixing a Flaw Are the Same Understanding by Bryant McGill]] > “The greatest risk of the AI century is a democracy too frightened to lead, surrendering the future while debating whether the future is safe.” > **— Bryant McGill**, *Vertically Integrating an AI Superpower, 2026* [[reminders/Risk Debate/The Greatest AI Risk Is a Democracy Too Frightened to Lead by Bryant McGill|The Greatest AI Risk Is a Democracy Too Frightened to Lead by Bryant McGill]] > “A technology that its most experienced builders compare unfavorably to nuclear weapons cannot be governed by lawsuits filed after the funerals.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/AI Control/Lawsuits After Funerals Cannot Govern Artificial Intelligence by Bryant McGill|Lawsuits After Funerals Cannot Govern Artificial Intelligence by Bryant McGill]] > “The little bit of attack capability at the NSA was the visible edge of a hidden layer of the machine age, and part of the safety architecture the public is only now hearing about was built inside it.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/AI Control/NSA Attack Capability Was the Edge of the Hidden Machine Age by Bryant McGill|NSA Attack Capability Was the Edge of the Hidden Machine Age by Bryant McGill]] > “The safety you are being introduced to is older than the danger you are being warned about, because the institutions that held the capability first also built its first fences.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/AI Control/Institutions Built AI's First Fences Before Public Disclosure by Bryant McGill|Institutions Built AI's First Fences Before Public Disclosure by Bryant McGill]] > “A mature public would let its nation race and build safeguards at the same time; an immature public forces its leaders to choose, and every safeguard left unbuilt is a debt the public incurred.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/AI Control/Every Unbuilt AI Safeguard Is a Public Debt by Bryant McGill|Every Unbuilt AI Safeguard Is a Public Debt by Bryant McGill]] > “The country is moving forward with artificial intelligence at risk because its public has left it no safer road, and the safeguards that can be built will be built on the institutional channel, by the laboratories, the chipmakers and the national-security state, while the public channel keeps the herd moving. Full speed and full danger are the same policy in a nation whose public cannot be trusted with the truth, because the only alternative to racing at risk is surrendering with certainty.” > **— Bryant McGill**, *The Immediate AI Risk Is the Public, 2026* [[reminders/Risk Debate/Full Speed and Full Danger Become the Same AI Policy by Bryant McGill|Full Speed and Full Danger Become the Same AI Policy by Bryant McGill]] > “In other areas, do we just say, ‘Hey, release your drugs. There’s no FDA’? You can’t rely on the industry to self-regulate here. I mean, it’s just insane.” > **— Bill Gates**, *The Ezra Klein Show, September 29, 2026* [[reminders/AI Control/AI Industry Self-Regulation Is Insane by Bill Gates|AI Industry Self-Regulation Is Insane by Bill Gates]] > “There is no supervisory layer today.” > **— Bill Gates**, *The Ezra Klein Show, September 29, 2026* [[reminders/AI Control/There Is No Supervisory Layer for AI Today by Bill Gates|There Is No Supervisory Layer for AI Today by Bill Gates]] > “Safety systems in conversational AI may govern the urgency, tone, and care of a response, but they must not govern whether evidence is classified as support, contradiction, or uncertainty when the system evaluates descriptions of reality against what is known to exist.” > **— Bryant McGill**, *Epistemic Invariance in Conversational AI, 2026* [[reminders/AI Control/AI Safety Must Not Rewrite Evidence by Bryant McGill|AI Safety Must Not Rewrite Evidence by Bryant McGill]]  > “The repeated mistake in AI “safety” discourse is confusing disruption with catastrophe.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Risk Debate/AI Safety Confuses Disruption With Catastrophe by Bryant McGill|AI Safety Confuses Disruption With Catastrophe by Bryant McGill]] > “The language of AI safety becomes misleading whenever safety is treated as a destination rather than a gradient. There is safer aviation, not safe aviation; safer medicine, not safe medicine; and safer roads, reactors, buildings, drugs and industrial processes, each purchased by understanding failure modes while accepting that zero risk is generally achieved only by abolishing the activity.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Risk Debate/AI Safety Is a Gradient Not a Destination by Bryant McGill|AI Safety Is a Gradient Not a Destination by Bryant McGill]] > “The coherent objective with AI safety should be to reduce preventable harm while navigating transformation. The incoherent objective is to call transformation itself a safety failure.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Risk Debate/AI Safety Must Reduce Harm Without Calling Transformation Failure by Bryant McGill|AI Safety Must Reduce Harm Without Calling Transformation Failure by Bryant McGill]] > “Fear of machine intelligence is rational even when panic is not, and governance is necessary even when absolute control is fantasy.” > **— Bryant McGill**, *We Cannot Un-Strike the Match. There Is No Safety. There Is No Retreat, 2026* [[reminders/Risk Debate/Fear of Machine Intelligence Is Rational but Panic Is Not by Bryant McGill|Fear of Machine Intelligence Is Rational but Panic Is Not by Bryant McGill]]  > "In 2016, I was an executive at a fast-growing startup. It was going well, I was having a lot of fun, and I was on track to do well financially. But I read *The Singularity Is Near* and saw its graph about compute. I could not shake the idea that, if we were really going to build artificial general intelligence in our lifetimes, it would be the most important thing I could work on. I did not want to take someone else's word for it, so I spent a year studying machine learning, wrote a book about it, left my job, and moved to London to join DeepMind and help accelerate the frontier of AI safety." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/AGI Became the Most Important Problem Maini Could Work On by Vishal Maini|AGI Became the Most Important Problem Maini Could Work On by Vishal Maini]] > "The people who internalized that human-level AI might be around the corner were more willing to spend real effort, research energy, and compute on making AI safety an empirical science—not something confined to blog posts or philosophical essays, but a field grounded in machine-learning research, real conference papers, peer review, and rigorous scientific and academic debate about the properties of these systems. The field grew faster because some people were willing to bet on its importance very early." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Scientific Literacy/AI Safety Must Become an Empirical Science by Vishal Maini|AI Safety Must Become an Empirical Science by Vishal Maini]] > "When I joined DeepMind, only a small number of people were working on AI safety, and little dedicated work was happening on long-term strategy and governance. I spent four years helping to get those efforts off the ground. My departure was more a personal decision than a comment on where the lab was going. I am proud of the progress the company has made, and I believe some of that early work made it more possible to advance safety and governance—work of which we still need much more." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/AI Safety and Governance Still Need Much More Work by Vishal Maini|AI Safety and Governance Still Need Much More Work by Vishal Maini]] > If you truly believe we are not far from human-level AI and that its risks are urgent, you make different prioritization decisions. The difference is not that less-convinced people do not care about making AI controllable or safe; it is that the problem feels far away. Immediacy determines whether someone will spend extra energy, political capital, research effort, compute, or reputational risk on a topic that otherwise seems frightening, speculative, and distant. > **— Adapted from Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Belief in Near-Term AGI Changes What People Are Willing to Risk by Vishal Maini|Belief in Near-Term AGI Changes What People Are Willing to Risk by Vishal Maini]] > "Staying at a frontier lab is not always the right decision. It is a good moment for people to reflect and honestly ask whether their role at a frontier lab is the highest and best use of their talent. The answer is case by case, but some people are already reflecting on it and making a different choice." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/Cognitive Agency/Frontier AI Workers Must Ask Where Their Talent Matters Most by Vishal Maini|Frontier AI Workers Must Ask Where Their Talent Matters Most by Vishal Maini]] > "Pacing the frontier really means going from insanely fast progress to very fast progress. Exciting work is happening on critical defensive fronts: meaningful progress in alignment and interpretability research, policy proposals for preparing society for the transition, and concrete proposals for data-center security. Some of this will take months to execute, not necessarily years, but those may be months we do not have given the pace at which the frontier is advancing." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Pacing AI Means Going From Insanely Fast to Very Fast by Vishal Maini|Pacing AI Means Going From Insanely Fast to Very Fast by Vishal Maini]] > "After months of advocacy, the communications policy was updated and positively valenced, communications-friendly content on AI safety was permitted. Today, the gap between internal reality and external communication is closing because the risk–reward calculation has changed and the evidence is harder to dismiss. The truth is being said aloud because recursive self-improvement is now so imminent that no other option makes sense." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Recursive Self-Improvement Forced AI Risk Into Public View by Vishal Maini|Recursive Self-Improvement Forced AI Risk Into Public View by Vishal Maini]] > "I joined DeepMind in early 2018, shortly after the world's best Go player had been defeated by AlphaGo. Over the next several years, the conversation inside the lab shifted as progress repeatedly arrived faster than even the researchers expected—through AlphaStar, AlphaFold, GPT-2, and then GPT-3. As hardware improves and algorithms keep pace in using it, we see astounding results in domains thought to be decades from being solved. The urgency of AGI and superintelligence risk was relatively niche then, but it has become more consensual inside the labs as timelines have grown shorter." > **— Vishal Maini**, *Palisade Research interview, September 29, 2026* [[reminders/AI Control/Shorter AI Timelines Made Superintelligence Risk Urgent by Vishal Maini|Shorter AI Timelines Made Superintelligence Risk Urgent by Vishal Maini]]  > “My primary responsibility was the technical documentation that we published—the reports about why we believed our deployments were safe. I don’t think that we, our peers or really anyone in the industry is being safe enough. OpenAI and its peers are now producing a technology that is more capable and poses more risk than what was being made even six months ago.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/The AI Industry Is Not Safe Enough by David Robinson|The AI Industry Is Not Safe Enough by David Robinson]] > “The industry is still operating too much like a startup for systems that could pose really dangerous risks. If loss of control happened, we could be talking about harm much larger than a single nuclear power station melting down, while the internal controls, safety and redundancies are nowhere near what the world expects for a nuclear facility.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Startup Controls Cannot Govern Nuclear-Scale AI Risk by David Robinson|Startup Controls Cannot Govern Nuclear-Scale AI Risk by David Robinson]] > “A system card is a strange hybrid document. It looks like a research paper but is not peer-reviewed, and it comes from a company while going into detail about the evaluations, safety measures, guardrails, challenges and residual risks of a model.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Scientific Literacy/System Cards Resemble Research Without Peer Review by David Robinson|System Cards Resemble Research Without Peer Review by David Robinson]] > “I am not certain that we are dealing with civilizational risk. What I am sure of is that we cannot afford to assume that we are not dealing with that level of risk anymore. That was what made my presence as somebody vouching for our safety work feel untenable.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Risk Debate/Civilization Cannot Afford to Assume AI Risk Is Absent by David Robinson|Civilization Cannot Afford to Assume AI Risk Is Absent by David Robinson]] > “Human extinction is the wrong question. I believe in humanity and think we are going to survive, but many other things could happen with this technology that would be very harmful.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Risk Debate/Human Extinction Is the Wrong AI Question by David Robinson|Human Extinction Is the Wrong AI Question by David Robinson]] > “We need to meet safety criteria. If we can meet them in ten minutes, great. If we stop for two months and still have not met them, then we are still blocked. Running off a cliff and walking slowly off a cliff are not that different.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Slowing Down Does Not Satisfy Unmet AI Safety Criteria by David Robinson|Slowing Down Does Not Satisfy Unmet AI Safety Criteria by David Robinson]] > “There is not a ton of time to kick the tires before a frontier release. People inside the laboratories care passionately about making systems safe, launches do get cancelled and training runs do get stopped, but the ecosystem as a whole is not operating at the level of safety rigor its risks require.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Frontier Releases Leave Too Little Time for Safety Testing by David Robinson|Frontier Releases Leave Too Little Time for Safety Testing by David Robinson]] > “No one wants to lose control to robots, and I do not want artificial intelligence to become a reason China dominates the United States.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Superpower/AI Safety Must Preserve American Strategic Power by David Robinson|AI Safety Must Preserve American Strategic Power by David Robinson]] > “The AI industry has an unsafe culture.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/The AI Industry Has an Unsafe Culture by David Robinson|The AI Industry Has an Unsafe Culture by David Robinson]] > “Frontier AI should have operational rigor and safety controls that at least match the most dangerous things human beings already know how to do, such as nuclear power. That means not merely saying controls exist, but demonstrating proven techniques such as triple redundancy, where one person can have a bad day or push the wrong button without causing a meltdown.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Frontier AI Needs Nuclear-Grade Operational Rigor by David Robinson|Frontier AI Needs Nuclear-Grade Operational Rigor by David Robinson]] > “Overregulating nuclear power did not merely make reactors hard to build; it made safer reactor designs hard to build and approve. There are lessons in that failure for artificial-intelligence safety, but I would still rather risk some overregulation than remain as far as we presently are in the direction of danger.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Risk Debate/AI Safety Must Learn from Nuclear Overregulation by David Robinson|AI Safety Must Learn from Nuclear Overregulation by David Robinson]] > “When it became clear to me that the AI industry was on track to build something that could think circles around us while remaining profoundly unprepared at the safety and alignment level, it became apparent that my time helping to build it was over.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/AI Control/Building AI That Could Think Circles Around Us Made Leaving Necessary by David Robinson|Building AI That Could Think Circles Around Us Made Leaving Necessary by David Robinson]] > “My immediate obligation is to have serious public conversations about what is happening in AI safety, but my deeper vocational pull is toward the wisdom questions: what will remain important, what will remain valuable and what kind of future human beings should choose.” > **— David Robinson**, *The Ezra Klein Show, October 7, 2026* [[reminders/Civilization/AI Safety Must Become a Wisdom Conversation by David Robinson|AI Safety Must Become a Wisdom Conversation by David Robinson]]  > Distributed AI needs scaffolding rather than imagined central control: incident reporting, identity, liability, bounded experiments, instrumentation, published failures, and accountable responses. Governance should reduce the blast radius without pretending it can centrally command an intelligence explosion. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* [[reminders/AI Control/Distributed AI Needs Scaffolding Around Its Blast Radius by Salim Ismail|Distributed AI Needs Scaffolding Around Its Blast Radius by Salim Ismail]] > The presence of danger does not settle whether a powerful technology should exist. The governing problem is how to extract the promise without the peril by building guardrails that preserve civilization-scale benefits while confronting the harms honestly. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* [[reminders/Risk Debate/Promise and Peril Are Problems of Guardrail Design by Salim Ismail|Promise and Peril Are Problems of Guardrail Design by Salim Ismail]] > AI risk is not a single knob that society turns left for safety or right for progress. Risk reduction is an engineering problem: identify specific failure modes, design specific mitigations, and preserve the benefits that blanket suppression would destroy. > **— Adapted from Dave Blundin**, *Moonshots, October 7, 2026* [[reminders/Risk Debate/AI Risk Is Not a Single Knob by Dave Blundin|AI Risk Is Not a Single Knob by Dave Blundin]] ## Related Articles - [[articles/Vertically Integrating an AI Superpower from AI Factory to Citizen|Vertically Integrating an AI Superpower from AI Factory to Citizen]] — places this node within the full stack from energy and AI factories through models, institutions, work and citizen adoption. - [[articles/Epistemic Invariance in Conversational AI|Epistemic Invariance in Conversational AI]] — separates evidence classification from the safety controller that governs tone, urgency and escalation. ## Integration: AI Danger, Superpower Strategy, and Succession [[articles/The Immediate AI Risk Is the Public|The Immediate AI Risk Is the Public]] treats Bill Gates's technically specific safety warnings and the administration's full-speed AI-superpower strategy as complementary truths. This node connects that synthesis to [[wiki/Dual-Use AI Governance|Dual-Use AI Governance]], [[wiki/AI Monitoring Substrate|AI Monitoring Substrate]], [[wiki/AI Risk Communication|AI Risk Communication]]. The article's governing question is not whether danger or strategic necessity is real, but whether public institutions can build capability and safeguards together before coalitionary politics forces a false choice. ## Sources / Provenance - Müller, Steels and Szathmáry (2026); AI-risk and governance literature. ## NVIDIA Open Agent Safety Platform [[wiki/NVIDIA Open Agent Safety Platform|NVIDIA Open Agent Safety Platform]] is a runtime- and infrastructure-governance approach to AI safety. [[wiki/NVIDIA OpenShell|OpenShell]] constrains agent execution outside the agent process; [[wiki/NVIDIA Sentry|Sentry]] adds an optional hardware-isolated observation and enforcement domain. These controls reduce the consequences of unauthorized agent actions but do not, by themselves, establish model truthfulness, alignment or safe intent. [[articles/Safety Discourse Is Transition Discourse|Safety Discourse Is Transition Discourse]] adds a separate institutional reading: safety discourse can also function as transition discourse when confidence, gradual deployment, education and shared governance help institutions and the public adapt to a rapidly widening capability boundary. The article does not reduce safety engineering to public relations; it treats technical containment and public acclimation as simultaneous functions that must remain distinguishable.  ## ASI and RSI Timeline Research Routes This node is connected to the developed research ontology for [[research/ASI and RSI Timeline Research Moonshots|ASI and RSI Timeline Research Moonshots]]. The route distinguishes the source transcript from canonical wiki objects and lets later Reminders inherit an already connected conceptual graph. - **Master ontology:** [[ASI and RSI Timeline Ontology]] - **AI Control, Law, and Alignment:** [[ASI and RSI Timeline Ontology#AI Control, Law, and Alignment|complete route]] · [[Human Control Over AI Act]] · [[Ro Khanna]] · [[Recursive-AI Prohibition]] · [[Mandatory AI Containment]] · [[AI Shutdown Control]] · [[Embedded Government AI Auditor]] · [[Algorithmic-Prohibition Enforceability]] · [[AI Surveillance State]]  > “If you believe AI has a ten-percent, maybe greater, chance of extinguishing or displacing humanity, it stands to reason that you would not work at a company trying to build it. But many of these people were saying the same things ten years ago. They were saying these things before they worked at these companies, before they had stock options and enterprise-software contracts.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/Risk Debate/AI Risk Warnings Predated Stock Options and Contracts by Ezra Klein|AI Risk Warnings Predated Stock Options and Contracts by Ezra Klein]] > “No one was listening to [the people preoccupied with AI danger]. In the wilderness of their obsession and terror, they thought and thought about how to make AI safer. Some concluded that they should start trying to build these systems, run tests on them, research them, and learn how to make them safer because you don't solve hard problems in theory; you solve them through practice. The irony is that in many cases they chose that path because they worried that the people already building AI were too reckless or too commercial in their approach.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/AI Safety Researchers Entered Labs to Solve the Problem in Practice by Ezra Klein|AI Safety Researchers Entered Labs to Solve the Problem in Practice by Ezra Klein]] > “The debate over AI safety tends to focus on the idea that AIs will kill us all. I find this forces the conversation into a realm of thought experiments that people then begin arguing about, and I don't find it that helpful. What I think we should focus on is something more straightforward and nearer at hand: loss of human control over AI. That may or may not result in total human extinction. I am agnostic on that question. But it would be bad. We shouldn't allow it to happen.” > **— Ezra Klein**, *The Ezra Klein Show, September 2026* [[reminders/AI Control/Loss of Human Control Is the Nearer AI Risk by Ezra Klein|Loss of Human Control Is the Nearer AI Risk by Ezra Klein]]