# Promise and Peril Are Problems of Guardrail Design by Salim Ismail > The presence of danger does not settle whether a powerful technology should exist. The governing problem is how to extract the promise without the peril by building guardrails that preserve civilization-scale benefits while confronting the harms honestly. > **— Adapted from Salim Ismail**, *Moonshots, October 7, 2026* ## Sources and Context - **Recording and precise locator:** [44:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2694s) — *The Fight Over Claude's Consciousness, AI's 1942 Moment, & Why Altman Says "Accept Some Bad Things"*, Moonshots, October 7, 2026. - **Exact reconciled source excerpt:** “The big challenge, as Neil Jacobstein puts it, is how do you extract the promise without the peril, right? Cars kill people; it doesn’t mean we get off cars. Electricity electrocutes people. You don’t ban electricity. You have to figure out the guardrails around this and figure out how you navigate the benefit and the promise versus the peril at civilization scale.” - **Editorial status:** The poster is explicitly labeled **Adapted from** because it consolidates spoken context into a self-contained proposition. It must not be represented as verbatim speech. Numerical claims, forecasts, legal interpretations, and claims about current organizations remain the speaker’s assertions unless separately verified. - **Reconciled source dossier:** [[research/All Things Superintelligence Research - The Coming Manhattan Project - Moonshots|All Things Superintelligence Research — The Coming Manhattan Project — Moonshots]] preserves broadcast order, speaker attribution, timing, and the source-fidelity boundary. - **Preparation ledger:** [[research/All Things Superintelligence Research - The Coming Manhattan Project - Moonshots - Reminder and Wiki Preparation|Reminder and Wiki Preparation]] records candidate 14, its mechanism, evidence posture, and misreading risks. ## Related Articles and Collections - **Collection:** [[collections/Simple Reminders|Simple Reminders]] - **Collection:** [[collections/Machine Succession|Machine Succession]] - **Wiki source hub:** [[wiki/Moonshots - The Coming Manhattan Project|Moonshots — The Coming Manhattan Project]] ## Related Topics - [[wiki/Salim Ismail|Salim Ismail]] - [[wiki/Dual-Use Technology|Dual-Use Technology]] - [[wiki/AI Governance|AI Governance]] - [[wiki/AI Safety|AI Safety]] ## Share on Social Media ``` The presence of danger does not settle whether a powerful technology should exist. The governing problem is how to extract the promise without the peril by building guardrails that preserve civilization-scale benefits while confronting the harms honestly. — Adapted from Salim Ismail, Moonshots, October 7, 2026 https://bryantmcgill.com/simple-reminders-salim-ismail-promise-and-peril-are-problems-of-guardrail-design ```