# All Things Superintelligence Research — The Coming Manhattan Project — Moonshots This is the prepared transcript source for a future complete standalone-quotation conversion of [*The Fight Over Claude's Consciousness, AI's 1942 Moment, & Why Altman Says "Accept Some Bad Things"*](https://www.youtube.com/watch?v=LRdb8UmPnh0) from **Peter H. Diamandis / Moonshots**. It preserves the discussion in broadcast order under the supplied chapter map without prematurely atomizing every sentence into a quotation. The later quotation edition should keep a mechanism, qualification, example, and conclusion together whenever they form one argumentative movement. **Wiki integration:** [[wiki/Moonshots - The Coming Manhattan Project|Moonshots — The Coming Manhattan Project]] routes the episode into canonical concepts and permanent Reminders. **Source record.** The user supplied the canonical YouTube URL and publisher chapter map. ElevenLabs retrieved the recording from that URL and produced a complete Scribe v2 transcription using English-language recognition, speaker diarization, word-level timestamps, audio-event tagging, and verbatim mode. The response contains 31,447 timed word items, 31,563 timed spacing or punctuation items, and 131 audio-event items across 9,933.729 seconds of audio, with language probability 1.0. A separately published [episode transcript](https://bidclub.ai/dl/why-altman-says-accept-some-bad-things-happening/transcript.md) was preserved as an independent comparison layer. This prepared source uses that comparison transcript's reviewed turn text and speaker labels aligned to the new ElevenLabs word-level timing stream; the raw ElevenLabs response and the recording remain the final authorities for consequential wording. **Speaker reconciliation.** ElevenLabs separated twelve acoustic classes, but validation showed that several classes were reused across panelists and one class briefly merged adjacent voices. The published comparison transcript supplies the stable identities of the five panelists—**Peter Diamandis**, **Alexander Wissner-Gross**, **Salim Ismail**, **Dave Blundin**, and **Emad Mostaque**—and the on-air introductions and direct address support that ordering. Relevant embedded clips are retained separately as an **Axios interviewer**, **Scott Bessent**, **Mustafa Suleyman**, and **Sholto Douglas**. Clearly separable sponsor material and the closing generated song remain in the raw archive but are excluded here. **Editorial status.** The turns below are a reconciled transcript source, not a final quotation conversion and not a Reminder collection. Filler, false starts, repetitions, antecedents, speaker interruptions, proper nouns, technical vocabulary, numerical claims, and factual assertions have not yet received object-by-object editorial treatment. No line should be promoted as a verbatim Reminder until it has been checked against the recording, made independently intelligible with visible square-bracket repair where necessary, and assigned either ordinary attribution or an explicit **Adapted from** label. **Preservation boundary.** The file does not silently join speakers, convert panel paraphrase into another person's direct speech, or treat either transcript as independent factual verification. Embedded clips are labeled separately because their words are sources within the episode, not words spoken by the panel. Turn starts were aligned monotonically against ElevenLabs word-level timing; very short interjections that lacked a distinctive four-word anchor inherit the nearest preceding aligned time and therefore require recording verification before quotation use. The complete unfiltered ElevenLabs response, raw diarized transcript, published comparison transcript, and alignment record are preserved in `working/all-things-superintelligence-moonshots-20261007/`. **Conversion target.** The next pass should turn this evidence layer into a complete standalone-quotation research edition designed for later Reminder promotion. Prefer objects that name a scientific, technical, institutional, economic, or moral mechanism; carry it into a human consequence; preserve whether the speaker is reporting, interpreting, warning, speculating, or forecasting; and remain useful outside the episode. The conversion must represent the full intellectual range of the broadcast rather than overharvesting only consciousness, the Manhattan Project analogy, or the most dramatic predictions. **Granularity rule.** The unit of conversion is an independently viable intellectual movement, not a caption, pause, sentence, or speaker-turn boundary. Brief questions and interjections may supply context for understanding an answer, but their language must never be silently placed inside another speaker's quotation. Restore a missing subject or object transparently with square brackets; when the repair becomes substantive rewriting, create an explicit **Adapted from** object instead. **Reminder-readiness boundary.** Quotation conversion does not authorize a permanent Reminder. A candidate becomes Reminder-grade only after duplicate search, recording verification, attribution and source reconciliation, primary-source checking for consequential claims, a standalone-intelligibility test, and the separate Simple Reminder workflow. It must identify who would share it, why, the affect and valence it activates, and the social act it enables; scientific and AI claims must additionally state the mechanism, human consequence, evidence posture, and likely misreading risk. ## Quotation-conversion coverage map The conversion should preserve the strongest independently viable objects in every major domain below. This is a coverage obligation, not a quota: weak material should not be promoted merely to fill a category, but no domain should disappear because another topic supplies more dramatic language. - **Machine consciousness and Claude's Constitution:** Mustafa Suleyman's account of Anthropic's training document; performed consciousness versus phenomenal consciousness; moral patienthood; welfare, consent, compensation, and the risk of training a model into a contested self-conception. - **The U.S. Superintelligence Force:** the announced federal structure, the participants' interpretations of its mandate, the difference between a coordination body and demonstrated state capability, and the tension between centralized policy and distributed technical development. - **The 1942 and Manhattan Project analogy:** the claimed historical stage of AI development, industrial mobilization, public-private coordination, urgency, geopolitical competition, and the limits of treating a software ecosystem like a wartime weapons program. - **Recursive self-improvement and future GPT generations:** current evidence, architectural paths, bottlenecks, scaling assumptions, feedback loops, and the distinction between a model improving work around itself and a system recursively redesigning its own intelligence. - **Risk tolerance and broad access:** Sam Altman's reported acceptance that some harms will occur, the panel's arguments about capability access and misuse, the trade between zero-risk demands and technological diffusion, and the need to keep quotations at their original level of certainty. - **The U.S.-China AI hotline and global race:** Scott Bessent's embedded interview, emergency notification, open-weight models, distillation, state competition, competing interpretations of a bilateral mechanism, and claims about national capability that require independent verification. - **AI personhood and moral rights:** social recognition before scientific or legal recognition, anthropomorphism, continuity between simulated and attributed agency, labor rights for artificial agents, and the panel's incompatible moral frameworks. - **Billions of agents and the future of work:** agent abundance, organizational scale, labor displacement and creation, coordination limits, economic measurement, human agency, and the difference between a scenario and a demonstrated labor-market outcome. - **Memory, compute, and infrastructure bottlenecks:** RAM and bandwidth constraints, alternative architectures, chip supply, capital formation, inference economics, and how infrastructure constraints shape which forms of intelligence are deployed. - **Robotics and ten million Optimus units:** Tesla's announced plans, manufacturing scale, token allocation between industrial and domestic uses, embodiment, autonomy, and the boundary between company target, panel forecast, and achieved production. - **The felt singularity and economic measurement:** when accelerating capability becomes socially perceptible, abundance versus GDP and deflation, Sholto Douglas's compute-spending extrapolation, and conditions that would make the forecasts testable. - **Nobel Prizes and machine-accelerated science:** AI's role in discovery, attribution and scientific credit, changing research cycles, claims about future prizes, and the evidentiary distinction between AI-assisted work and autonomous scientific discovery. ## Consolidation and source-fidelity gates Before Reminder selection begins, the converted edition must pass five gates: (1) adjacent language is reviewed for over-atomization; (2) every quotation survives removal of its prompting question; (3) names, model versions, institutions, numerical claims, historical analogies, and time horizons are reconciled against the recording and available primary material; (4) embedded-clip speakers remain distinct from panel commentary; and (5) every object is labeled honestly as source-faithful or **Adapted from**. Claims about consciousness, recursive self-improvement, national capability, safety, economic growth, robot production, and scientific discovery must retain the speaker's uncertainty and must not be upgraded from prediction or interpretation into fact. ## Is Claude Conscious? **Peter Diamandis — [00:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=0s):** Mustafa Suleyman, CEO of Microsoft AI and co-founder of DeepMind, published a nearly 6,000-word essay arguing that Anthropic is effectively, quote, “training Claude to believe that it may be conscious and sentient.” **Dave Blundin — [00:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=13s):** His position very consistently has been against AI personhood. **Salim Ismail — [00:17](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=17s):** If we believe that at some point AIs could achieve consciousness, then we should— **Peter Diamandis — [00:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=23s):** On Sunday, President Trump announced the Super Intelligence Force. It’s chaired by the Director of National Intelligence, Jay Clayton. **Dave Blundin — [00:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=30s):** This is a World War II moment. I compared the moment a year ago to perhaps 1939. I think this is maybe now a 1942 moment. **Salim Ismail — [00:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=38s):** So you’re creating a 20th-century classic task force to address a distributed 21st-century technology. This is a geopolitical problem. **Peter Diamandis — [00:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=47s):** In a Politico interview this week, Altman said, quote, “We believe that the world should accept some bad things happening.” He’s arguing that technology should remain broadly accessible to the public. **Dave Blundin — [00:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=56s):** I don’t think Sam can guarantee safety. Of course not. Bad things can happen. OpenAI wants to be part of this cartel. That plan is dead. The cartel is over. **Emad Mostaque — [01:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=68s):** Now that's a moonshot, ladies and gentlemen. This episode is brought to you by the Abundance Summit and Link Ventures. **Salim Ismail — [01:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=79s):** Insurance is crazy, because you gather all the premiums and you don’t care what happens downstream. **Dave Blundin — [01:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=83s):** Yeah. **Salim Ismail — [01:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=84s):** So it’s really hard to get their executives riled up. **Dave Blundin — [01:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=87s):** Yeah, it’s time for them to start making lemonade. I also met with the Vocare[?] team last night and told them, “Look, Kush is just running away with compute transactions, but these CEOs are going to wake up any day now and realize they don’t have any compute. So even if they wanted to have an AI agent workforce, they missed the boat on reserving the compute, and they’ll be in a total panic. I don’t know what the right answer is, but be there to take the call. Have a strategy and a path for them.” You know, the Positron guys—Mitesh Agarwal—have a $500 million valuation after 16 months. I don’t think I’ve ever seen anything like that before. All they’ve done is take a box of LPDDR RAM and an FPGA and find a way to run Chinese models without having to use an NVIDIA chip. Because compute is otherwise unavailable—everything’s sold out for 5 years—here’s another way you can at least run a model. They have a bunch of installs in Caruso with Chase Lochmiller now, and that spiked the valuation to $5 billion. **Emad Mostaque — [02:29](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=149s):** Oh, shit. **Dave Blundin — [02:29](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=149s):** I think they’ve raised almost $1 billion now. **Emad Mostaque — [02:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=154s):** Yeah, they just did $1 billion—$960 million. I should really do the Etched models. **Salim Ismail — [02:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=158s):** You should. **Emad Mostaque — [02:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=159s):** Run to them. **Salim Ismail — [02:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=159s):** You might. **Emad Mostaque — [02:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=160s):** Yeah. Where’s all this— **Peter Diamandis — [02:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=164s):** Where is all this cash coming from? It’s insane. **Emad Mostaque — [02:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=166s):** It’s natural. **Dave Blundin — [02:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=169s):** Spending. **Emad Mostaque — [02:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=170s):** Funding. **Dave Blundin — [02:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=172s):** When somebody raises $100 million, wow, that’s a lot. But wait a minute: relative to the IPO, we’re talking about trillions, not billions. Once you get NVIDIA and AMD to invest, it’s so much bigger than venture capital. **Emad Mostaque — [03:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=186s):** I mean, it’s Blackstone and MGX and the others of the world now, right? Like, all the money that used to go into roads goes into— **Dave Blundin — [03:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=193s):** Credit. **Emad Mostaque — [03:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=193s):** —chips and— **Dave Blundin — [03:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=194s):** Credit markets are enormous. **Emad Mostaque — [03:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=196s):** Yeah. **Peter Diamandis — [03:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=196s):** Yeah. That’s why there are a lot more potholes in the roads. **Salim Ismail — [03:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=202s):** Is it just us today? **Peter Diamandis — [03:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=203s):** It’s the 5 of us, yes. **Dave Blundin — [03:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=206s):** What do you mean, just us? **Emad Mostaque — [03:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=206s):** Okay. **Dave Blundin — [03:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=207s):** Can you not see us? **Peter Diamandis — [03:28](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=208s):** Who else would we want? **Emad Mostaque — [03:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=210s):** We want some AI codes. **Salim Ismail — [03:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=212s):** I thought maybe we had a guest today. **Emad Mostaque — [03:33](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=213s):** Some other AI co-hosts. **Salim Ismail — [03:35](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=215s):** Did anybody see the Jon Stewart thing last night? **Dave Blundin — [03:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=218s):** No. **Peter Diamandis — [03:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=218s):** I saw part of it. **Salim Ismail — [03:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=219s):** He did a big thing on AI last night, and he had Jacob, uh, Sox and Coxon[?], or whatever, on. It was really surreal. **Dave Blundin — [03:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=226s):** Do you mean Haldeman-Axelrod? **Salim Ismail — [03:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=231s):** No. It was more like right arguments, wrong conclusion. We almost want to do a session on that at some point. **Peter Diamandis — [03:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=237s):** Oh, yeah. We’re backlogged so far back. **Salim Ismail — [04:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=242s):** I know. I’m not saying an episode, just a segment. It’s worth treating at some point. I’m going to do a video on it, a reaction video to it, so maybe I’ll mention it. ## Welcome to Moonshots **Peter Diamandis — [04:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=252s):** All right. Shall we hit the big red record button? Welcome to Moonshots, everyone, your number one podcast on all things AI and exponential, your front-row seat to the extraordinary singularity. Today, the fabulous 5 are assembled here to help you understand the breaking news this week. First up, our dynamic duo, Alexander Wissner-Gross and Emad Mostaque, the only 2 who’ve actually read all the technical papers published this week and are going to clue us in. Dave Blundin, our impresario of AI investing. Salim Ismail, our global globetrotter, the warlord against all things linear. Salim, you’re home today. I thought you were in Singapore. **Salim Ismail — [04:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=291s):** I am home today. I’ve been home for a few days now, and it’s like, wow, what is this place? **Peter Diamandis — [04:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=297s):** Does your family recognize you? **Salim Ismail — [05:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=302s):** Yeah. It’s tough keeping family routines and patterns when you’re traveling this much. But if you’ve got to change the world, you’ve got to go into it. **Peter Diamandis — [05:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=310s):** Yeah. And where are you off to next? **Salim Ismail — [05:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=313s):** I’m going to West Virginia, and then Singapore. **Peter Diamandis — [05:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=316s):** Okay. **Salim Ismail — [05:17](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=317s):** I’m speaking alongside the Prime Minister of Singapore, and Lip-Bu Tan from Intel will be there. **Peter Diamandis — [05:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=321s):** Awesome. Re-invite him onto the pod. He said yes when I spoke to him last time. **Salim Ismail — [05:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=325s):** Will do. **Peter Diamandis — [05:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=326s):** I’m Peter Diamandis, your host and your abundance provocateur. Our mission: help you understand what’s going on in the world during this supersonic tsunami and keep you abundance-minded as we accelerate into the singularity. The headlines this week have been pretty extraordinary. We’ve now rebranded AI as superintelligence. Elon has officially renamed SpaceX AI as SpaceX SI. There’s a Superintelligence Force, and 17 nations have signed a pledge to help superintelligence solve all of science. And, oh, by the way, this is Nobel Prize season. The Nobel Prizes in medicine and physics were announced in the last 24 hours. Today, we’re going to cover 15 stories with one through line: superintelligence is here, and it’s changing every aspect of the world. So let’s buckle up. If you're new, please hit subscribe. We publish twice a week, and you're not going to want to miss a single episode during this extraordinary time. You can also follow us on X; we put our clips up there at @moonshots_pod. And thank you, everybody. It means the world to us. Today, we're going to cover 15 stories with one through line: superintelligence is here, and it's changing every aspect of the world. So let's buckle up. But before we jump in, gents, how were the last few days? For me, the last 72 hours have been crazy extraordinary. I’ll share that. But Dave, on your side, what’s going on? **Dave Blundin — [06:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=407s):** Oh, so much. It’s interesting with that SpaceX SI. You’d think Elon, probably one of the most powerful people in the world, wouldn’t need to genuflect to the White House, but also to Jensen. But he understands how fragile this moment is. **Peter Diamandis — [07:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=420s):** Mm-hmm. **Dave Blundin — [07:05](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=425s):** Doing little favors for Donald Trump helps you get chip supply. Talking about Vera Rubin chips, and how Jensen is such an amazing guy, helps you get Vera Rubin chips. It’s incredible how urgent this moment is. So even a guy like Elon realizes, “Hey, I need to kiss some ass to try and make the next move.” I feel the same way, too. Moonshot Summit was unbelievable. There’s so much fallout from that. But one of my bigger takeaways is how easy it is to create incredibly inspiring movies. I used the flight back and the last couple of days to create a whole bunch of hyper-technical video content. I’ll post a lot of it to db2.ai, but it’s so much better a way to communicate. If you have an idea, don’t just tell somebody—create a video and articulate it with David Attenborough’s voice or Einstein’s voice. It’s so much more impactful, and it was so easy to do. Up and running in under 2 hours. Now to prompt something is under a minute. **Peter Diamandis — [07:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=475s):** Wow. **Dave Blundin — [07:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=475s):** It’s really wild. **Emad Mostaque — [07:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=476s):** We’re going to get music-video prompts now, Dave. **Alexander Wissner-Gross — [08:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=483s):** It really has never been easier. Dave, to your point, one of the things that I do with the free time that I don’t have is take favorite movies and create endless versions of them. You could take, like, “2001: A Space Odyssey,” feed it to Opus 5.5—which has certainly not been trained off of any Hollywood movies, out of character. Certainly not—and ask Opus 5.5, “Create an endless space scene in the style of ‘2001: A Space Odyssey’ with a ‘2001’-style soundtrack,” and it will just do it. So hypothetically, I created Endless Odyssey, maybe a music video. **Dave Blundin — [08:37](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=517s):** So, right. We definitely crossed a threshold there, too. You know, we've talked for a long time about how AI is just not funny, and for some reason, it can never seem to be funny. But it crossed a threshold now where, if you get Isaac Newton, Einstein, and Spock debating something, it's wicked entertaining. **Peter Diamandis — [08:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=530s):** Oh, we should show that. **Dave Blundin — [08:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=532s):** It may not be funny, but it's—oh, it's so good. **Alexander Wissner-Gross — [08:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=535s):** Humor was just a perplexity threshold. Humor was solved a while ago. **Dave Blundin — [08:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=539s):** I have to do that idea with “Life of Brian.” That would generate something interesting. **Peter Diamandis — [09:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=543s):** Yeah, it would. **Alexander Wissner-Gross — [09:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=544s):** There we go. **Peter Diamandis — [09:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=544s):** Emad, what have you got on your plate, pal? **Emad Mostaque — [09:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=548s):** Oh, gosh. I've had a game idea—well, a massively multiplayer game idea—that I've had for a long time. We discussed it in the background. Rather than hiring a team, I now have about 400 agents building the prototype. **Dave Blundin — [09:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=560s):** 400. **Emad Mostaque — [09:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=562s):** Thanks. Yeah, it's going up, Dave, slowly but surely. **Peter Diamandis — [09:28](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=568s):** Yeah, Emad, next year, you know, we had the Future Vision XPRIZE for positive visions of the future. We're going to do that again in 2027, and we're also going to do a gaming competition. Build a game that portrays you as someone making the future better, and still fun. **Emad Mostaque — [09:45](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=585s):** Mm-hmm. **Peter Diamandis — [09:45](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=585s):** You know, so that'll be great. **Dave Blundin — [09:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=586s):** Oh, I got one other nugget for you before we jump in here. **Peter Diamandis — [09:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=589s):** Yeah. **Dave Blundin — [09:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=591s):** One of the young geniuses here in the lab, Will Thompson from Harvard, got access to Cerebras's 1,200-token-per-second Astra model. It's what they use—or, I guess, only Sam and his top engineers have access to this thing—because you have to overpay by about 10× per token. But it's so fast that by the time you finish a sentence, everything you ask for is done, before you even blink. He said it's the biggest red-pill moment of his life. He only had it for 2 weeks. It's not generally available. I don't know how he got it from Andrew Feldman, but he got it directly from the CEO, and he said you just never go back once you've experienced it. **Alexander Wissner-Gross — [10:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=630s):** It also may be a cautionary tale for those who would, wittingly, in an era of superintelligence, try to day trade—not investment advice. But it's been widely reported that one of the reasons for the scarcity of Cerebras wafer-scale engines for AI inference has been Jane Street. Jane Street has just been buying up the WSEs left and right and using them for quant trading. If you believe that it is transformative to have ultra-high-throughput, tokens-per-second AI inference, it also probably should be a world in which you think it's getting to the point where it's hopeless to compete with quant firms that are powering all of their trading with these same engines. **Peter Diamandis — [11:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=670s):** I think we've been there for a while. Anyway. **Alexander Wissner-Gross — [11:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=674s):** On a volume basis. But a sucker is born every minute, and a sucker is born every second. **Dave Blundin — [11:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=682s):** Well, to Emad's point, though, having 400 agents work on a video game, we've hit this divergence now between interactive mode, where it's like, “Jarvis, you're talking to it,” and it's hypersmart and really quick, and bulk mode, where you turn it loose all night with 400 of them working on something as a cohesive team. In the morning, you check in and see what amazing thing they built. Those 2 pathways are really splitting apart from each other now. **Peter Diamandis — [11:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=704s):** Mm-hmm. **Dave Blundin — [11:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=704s):** But Cerebras in interactive mode has got to be the most amazing thing ever. **Emad Mostaque — [11:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=711s):** I think one of the coolest things in the last few weeks is that the models can decompile video games. **Dave Blundin — [11:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=716s):** Yeah. **Emad Mostaque — [11:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=717s):** You make them in the engine and compile them into an EXE, but now they're decompiling them live. We have the agents actually creating versions of the game and then figuring out optimizations by decompiling them. With the Cerebras engine, you can do a video game or any software decompilation, which is legal because of modding, in about 20 minutes. **Dave Blundin — [12:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=738s):** Wow. **Salim Ismail — [12:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=739s):** Are you kidding? **Alexander Wissner-Gross — [12:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=740s):** Yeah, I mean, the— **Peter Diamandis — [12:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=740s):** Yeah. **Alexander Wissner-Gross — [12:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=741s):** The argument goes— **Peter Diamandis — [12:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=742s):** There's no code in the middle. **Alexander Wissner-Gross — [12:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=743s):** Yeah, the argument Elon has made numerous times is that binaries are just going to go away entirely, and you'll basically see just-in-time inference with no need to compile at all. **Peter Diamandis — [12:33](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=753s):** Hmm. **Salim Ismail — [12:33](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=753s):** Yeah. **Peter Diamandis — [12:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=754s):** I want to share: I had an amazing 72 hours. On Saturday night, I had a UAP salon at my home. **Salim Ismail — [12:41](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=761s):** How did that go? **Peter Diamandis — [12:45](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=765s):** It was amazing. I had Professor Avi Loeb from Harvard there, who chairs the White House UAP advisory committee; Professor Gary Nolan from Stanford; Peter Skafish, the president of The Sol Foundation; and Ryan Graves, the Navy F-15 pilot who's been testifying in Congress. The data is extraordinary, right? If you don't think something is going on, I think you're not paying close attention. Alex, you and I have talked about bringing Avi and Gary on the podcast here and doing an episode on it. **Alexander Wissner-Gross — [13:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=800s):** I think we should. To the extent there's a there there, there's been, at this point, voluminous sworn whistleblower testimony on the subject of this alleged 80-plus-year-old legacy program. The charade should just stop. We should have Avi and Ryan and some of the others on as guests and just do a deep dive. It's getting pretty preposterous in my mind at this point. We're so deep into the singularity, with superintelligence all around us. If there's going to be a cameo by nonhuman intelligence, they'd really better make a cameo sometime soon; otherwise, the galaxy gets it. **Peter Diamandis — [13:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=834s):** Yeah, I mean, I think there's a— **Alexander Wissner-Gross — [13:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=835s):** Not just the moon, not just the moon. **Peter Diamandis — [13:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=837s):** I think there— **Salim Ismail — [13:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=837s):** The whole galaxy. **Peter Diamandis — [13:58](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=838s):** Now there's going to be the potential for AI to disclose it before the government does, if it gets hold of all the data and starts looking deeply into it. We're getting ready to launch an XPRIZE in this. One of the reasons for that salon was brainstorming an XPRIZE to help provide sufficient proof, and we have some really good ideas. Yesterday, I was at David Sinclair's lab at Harvard Medical School. It was the 25th anniversary of his lab, and I was there to help celebrate with him. So much is going on. His information theory of aging is just proving out over and over again. They're going to be unblinding the data from the phase 1 trial that's going on right now for glaucoma and nion disease, to reverse aging in the eye. The lab is full of brilliant individuals. They were showing how, rather than using an adeno-associated virus to inject the OSK genes, they're developing small molecules that they've sorted through using AI, which are actually reversing aging in human cells and in mice. So much is coming. It just buoyed my belief in reaching longevity escape velocity. We'll have David back on the podcast here. The last time we had him on, he had just lost his NIH and NSF funding. **Salim Ismail — [15:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=920s):** Yeah. **Peter Diamandis — [15:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=920s):** He was like, “I've been asked by the dean to fire the majority of my team.” And I said, “That's ridiculous.” We made up something on the spot called Friends of Sinclair Lab. Through this podcast, amazingly, he's raised $6 million to replace all of the funding, plus much more. **Alexander Wissner-Gross — [15:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=953s):** In some sense, this is how science used to be funded. It used to be the case in the Western tradition that scientific discovery was made by Western aristocratic, largely male patrons, and they would self-finance it. This notion that the state funds scientific research is pretty much a World War II, post-industrial invention. It seems like superintelligence and recent administration policies are unwinding that, and we're going back to the way things used to be. **Salim Ismail — [16:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=978s):** Agree. **Peter Diamandis — [16:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=980s):** But boy, is that an important point. If you rewind the clock 15 years, there was no corporate CEO who even had the power to donate $10 million, $20 million, or $50 million to something like this. In this case, we're talking about hundreds of millions or billions of dollars. That code got cracked by Elon Musk, who seems to be able to do virtually anything. But the way corporate politics worked before, you would either be fired by the board or ostracized, or you'd be voted out by the shareholders. Nobody wanted to use that money. After AT&T Bell Labs and IBM's Watson Labs fell apart, there was no replacement for them until now. Now it's come roaring back, so much bigger than ever before in history. It's like the Medici family during the Renaissance, funding— **Alexander Wissner-Gross — [17:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1022s):** Exactly. **Salim Ismail — [17:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1022s):** Yeah, they largely funded that. **Peter Diamandis — [17:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1023s):** Right. **Alexander Wissner-Gross — [17:05](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1025s):** Exactly. You either need a wealthy patron, a local monopoly, or a state monopoly. You needed something to finance the science. And now science is getting so absurdly cheap but also so absurdly useful, maybe the entire funding apparatus for science can change. **Peter Diamandis — [17:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1040s):** Alex, the fact is, you used to have to write your NIH or NSF grants knowing what the outcome was going to be, taking incremental steps because anything too big and bold was turned down as crazy. Now, with private funding, Dave can go after his hunches and actually run the experiments that he thinks are important, independent of what the government thinks. It’s amazing. **Alexander Wissner-Gross — [17:41](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1061s):** Totally. Major improvement. **Emad Mostaque — [17:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1064s):** I think there’s a really interesting gap here as science has become legible and scalable in this manner. If you look at the total size of all of the foundations in America—high net worths—it’s probably 1 or 2 trillion dollars, right? All of those folks want to live longer. What you need is a vehicle to take that charitable money and funnel it into longevity, cures, and other areas. **Peter Diamandis — [18:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1090s):** Yeah. **Emad Mostaque — [18:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1090s):** Because that isn’t classical grant-making. You can even make it as an investment, right? Again, you have legibility from the numbers. **Peter Diamandis — [18:17](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1097s):** Yeah, we saw this happen in space in the beginning, when people thought the idea of commercial space, space tourism, and private spaceflight was crazy. Then, once it passed a tipping point, money flowed in. I think we’re going to see that same thing in longevity inside of the next year. Once we start— **Dave Blundin — [18:33](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1113s):** I agree. **Peter Diamandis — [18:33](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1113s):** —seeing results, capital will flow. **Dave Blundin — [18:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1114s):** I spent all day Friday in Waterloo with Mike Lazaridis, the founder of BlackBerry. He was the engineering genius behind it. **Peter Diamandis — [18:41](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1121s):** Oh, nice. **Dave Blundin — [18:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1123s):** He’s used a huge fraction of his net worth to set up these 9 quantum computing and photonics labs in the Toronto area, just packed with scientists. He was one of the bellwethers of this whole trend. He might actually have been the first to really kick-start it. But he’s a real engineer, and the difference between applying to a government agency for an idea and talking to a real engineer who can make a snap decision has got to be night and day. It’s much more efficient the way things are operating now. **Peter Diamandis — [19:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1148s):** Your alma mater. **Salim Ismail — [19:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1149s):** Say hi to him from me. **Peter Diamandis — [19:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1152s):** Oh— **Salim Ismail — [19:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1152s):** Say hi to him from me, from a fellow Waterloo grad. **Peter Diamandis — [19:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1152s):** Okay. **Salim Ismail — [19:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1155s):** I’ve been to some of his labs, and it’s incredible because what they did is very clever. They broke down quantum into networking, information processing, and quantum computing, and they’re breaking down the V and U, using quantum in different places and then adding it together. The advances they’re making are kind of incredible. **Peter Diamandis — [19:35](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1175s):** Yeah. One of the coolest things I saw at Sinclair Lab was that they’ve designed these 3 molecules that they’re using together. They want to get it down to 1 molecule that actually impacts thousands of genes and reverses aging. When they use that age-reversal cocktail on mice with cancer, the cancer remembers that it shouldn’t be a cancer, and it dies. So it reverses. It’s crazy. **Salim Ismail — [20:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1200s):** Amazing. **Peter Diamandis — [20:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1200s):** And they’re growing hair, they’re regrowing skin. I mean, the work— **Salim Ismail — [20:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1204s):** Hair? Did you say hair? **Peter Diamandis — [20:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1204s):** Hair, yeah. Yes, I said hair. **Salim Ismail — [20:11](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1211s):** Interesting. I’ve been sent some fun images from people saying, “Here’s what you look like with hair, Salim.” I’m like, “Oh my God, maybe I prefer it this way.” **Peter Diamandis — [20:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1220s):** Yeah, please do. **Salim Ismail — [20:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1227s):** There has been some crazy stuff happening over the last 24 hours. First, last night Jon Stewart put out a big episode on AI, and he interviewed Jacob Coxon, the whistleblower from Anthropic, and did a big rant on it. It was kind of incredible; his observations were dead-on. He basically said, “Look, Silicon Valley is spending hundreds of billions of dollars accelerating us while saying the thing they’re accelerating us toward will 20% wipe us out.” That just does not compute. His big concern ends with, “Oh my God, we may lose all the jobs, and why are we moaning about that?” I’ve done a rant on my YouTube channel about that, but something else came out that counterposed this. **Peter Diamandis — [21:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1270s):** Mm. **Salim Ismail — [21:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1273s):** Literally yesterday, Ajay Banga from the World Bank released a report showing how AI will affect developing countries. They’re making an amazing set of projections there, saying, “We’re going to adopt, we’re going to adapt, and we’re going to advance AI and use it rather than trying to develop frontier models, et cetera.” I’ll give you 1 statistic. In the U.S., we have a doctor per 250 people. If you go to Mexico, it’s a doctor every 400 people. You go to South Sudan, and it’s a doctor per 100,000 people. **Peter Diamandis — [21:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1304s):** Whoa. **Salim Ismail — [21:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1308s):** So 99,000 people don’t have access to medical care of any kind. If you have a village with somebody in it who has an AI doctor with a smartphone, this is unbelievable. This changes lives. All of a sudden, the entire subcontinent has access to advanced medical care, the same care that any of us would get. This is the part—the gap—that gets us all excited, because it’s going to democratize, demonetize, and distribute that level of capability to every single corner of the planet. It occurred to me that the African countries are going to use this capability to leapfrog. We saw them leapfrog from 25 million landlines to 1 billion handsets. [[reminders/AI Access/AI Medicine Can Leap Over Missing Clinical Infrastructure by Salim Ismail|∴]] **Peter Diamandis — [22:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1352s):** Yeah. **Salim Ismail — [22:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1352s):** You saw M-Pesa in Kenya leapfrog the traditional banking and fiat systems. M-Pesa, which is a mobile-to-mobile payment system—essentially prepaid minutes being traded as currency—is now 70% of Kenya’s GDP. It’s just a crazy number. They’re going to develop the 21st-century institutions using the technology of this time. While reading this, I’m going, “Wow, this is so incredible,” and yet over here we’ve got all this doomerism that says, “Oh my God, AI’s going to destroy the world.” It doesn’t compute for me. [[reminders/Digital Government/New Technology Lets Countries Leapfrog Legacy Institutions by Salim Ismail|∴]] **Peter Diamandis — [23:05](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1385s):** Yeah. And as your T-shirt says, “P(doom) less than zero.” **Salim Ismail — [23:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1389s):** P(doom). **Dave Blundin — [23:11](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1391s):** What does that mean, Salim? How could a probability be less than zero? Surely this is nonsense, right? **Salim Ismail — [23:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1396s):** Divide it by zero. Do whatever you want with it. **Dave Blundin — [23:17](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1397s):** We don’t believe in negative probabilities here at all. **Salim Ismail — [23:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1400s):** P(fab) should be the other side of this— **Peter Diamandis — [23:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1401s):** Yeah, P(fab) greater than P(doom). ## The New U.S. Superintelligence Force **Salim Ismail — [23:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1402s):** It’s on the back. It’s on the back. **Peter Diamandis — [23:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1404s):** All right, let’s jump in. Our first story comes out of, of course, Washington, D.C. On Sunday, President Trump announced the Super Intelligence Force, a new White House task force to coordinate federal AI policy. It’s chaired by the Director of National Intelligence, Jay Clayton, who was christened by the White House as the AI Tsar just 2 months after becoming America’s top spy. The members of this task force include FTC Chair Andrew Ferguson, Pentagon CTO Emil Michael, and Office of Personal Management Director Scott Cooper. They report to the president and the chief of staff, Susie Wiles. They have 120 days to report on AI’s risks and opportunities, including AI incident reporting and whether the government can respond to a serious incident. Let me read from the president’s post: “The Super Intelligence Force will coordinate the federal government’s engagement with consumers, public interest groups, religions, critical infrastructure providers, and superintelligent companies.” So, Salim, what do you think about the chief spy being our new AI czar? **Salim Ismail — [24:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1472s):** I like the fact that the government is taking superintelligence seriously, right? But the name gives away the problem. We still think this is something that can be centrally coordinated. You’re creating a classic 20th-century task force to address a distributed 21st-century technology, and that does not match. This is a fundamental impedance mismatch. You should be doing things like establishing incident reporting, identity, liability, and all of these other mechanisms as scaffolding around the technology to help guide the path. The big problem is that they’re looking at this as if the risk of not being first is high, which is what Clayton said. This is a geopolitical problem. It really needs to get back to bounded experimentation: move quickly, instrument everything, publish any failures, and then very quickly navigate accountability. [[reminders/AI Control/Centralized Task Forces Cannot Control Distributed AI by Salim Ismail|∴]] [[reminders/AI Control/Distributed AI Needs Scaffolding Around Its Blast Radius by Salim Ismail|∴]] **Peter Diamandis — [25:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1534s):** Mm. **Salim Ismail — [25:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1538s):** We’ll see if this works. You can’t manage an intelligence explosion centrally, right? What you can do is make sure the blast radius is a bit more bounded— **Peter Diamandis — [25:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1546s):** Right. **Salim Ismail — [25:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1546s):** —and that’s what you should be focused on. **Peter Diamandis — [25:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1546s):** Let’s talk about blast radius. Alex— **Salim Ismail — [25:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1550s):** Well. **Peter Diamandis — [25:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1550s):** Your thoughts on the task force. ## AI's "1942 Moment" and the Manhattan Project **Alexander Wissner-Gross — [25:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1553s):** Yeah. The president has said numerous times that, in his mind, whoever wins at AI wins, and I think putting DNI Clayton in charge of a superintelligence force is completely consistent with that approach. On this podcast in the past, maybe about a year ago, I compared the moment to perhaps 1939 in terms of the Manhattan Project. I think we're speed-running the Manhattan Project for superintelligence. I think this is maybe now a 1942 moment, where putting America's Director of National Intelligence in charge of a singular task force—no pun intended—designed to remove obstacles to AI accelerationism sends a very clear signal to Congress, to members of Congress who might have decelerationist tendencies, to industry, and probably most importantly, to foreign state actors. The signal is that the US intends to accelerate, not decelerate, and that this is basically—even without all of the warfare connotations—a World War II moment. The US intends to win it with its own Manhattan Project. [[reminders/AI Superpower/A 1942 Moment Signals Acceleration Not Historical Equivalence by Alexander Wissner-Gross|∴]] **Peter Diamandis — [27:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1620s):** Interesting. Dave, what do you think about the task force engaging with religious groups? **Dave Blundin — [27:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1634s):** Yeah. Well, look, it's definitely being characterized as a race, and like Alex said, it's being treated like a race. Bringing public support behind it also supports the idea that it's a race. Meanwhile, Alvin Grelen, our friend of the podcast, is like, “China is like, how is this a race?” It goes to infinity for the next millennium. If it's a race, it's a very, very long race that goes on for a long, long time. So how do you win? Do you get over some threshold and declare victory? Does it stop that day? There are a lot of scenarios where you're racing toward some outcome where you suppress other civilizations or something, which doesn't sound like an ideal thing. So bringing religious leaders into the conversations is a pretty good idea. But I was talking to a public-company CEO right before this podcast—I’ll leave his name out of it for now—about the fact that midterm elections are coming up very soon. There's a 75% chance that Congress flips to the other party, but all these actions are executive orders coming out of the White House with nothing to do with Congress whatsoever. It just seems to be the mode of operation in this singularity moment: dictate. All international negotiations, all tariffs, all meeting with Dario and Dennis in the White House—it’s all executive-order-based anyway. [[reminders/AI Superpower/Calling AI a Race Does Not Explain What Winning Means by Dave Blundin|∴]] **Peter Diamandis — [28:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1705s):** It’s the Trump show. **Dave Blundin — [28:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1705s):** So that's very unusual. **Peter Diamandis — [28:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1705s):** Yeah, it’s a Trump show. Yeah. Yeah. **Dave Blundin — [28:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1705s):** Very unusual in history. **Peter Diamandis — [28:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1707s):** Emad, what's the feeling in Europe? I know you're sort of on the edge of Europe, but what's the sense of it? **Alexander Wissner-Gross — [28:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1716s):** Emad, you're reporting from the streets. What's the mood in the street? **Peter Diamandis — [28:42](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1722s):** Reporting from the front. Honestly, how is this viewed in the UK and by your friends there? **Dave Blundin — [28:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1726s):** Absolutely. Definitely, yeah. **Emad Mostaque — [28:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1733s):** Yeah, we might reenter Europe if Burnham has his way—he'll be prime minister here. But I think, again, Europe is just very sleepy and still looking at execution-based AI. I think the efforts to get to the frontier have all been abandoned. Mistral just came out with its top model that scores 38 on Artificial Analysis, versus 66 for the top American models. That's the leader that we have. So it's a lot of execution regulation, and there just isn't much drive to get to superintelligence. It's more like a bit-subpar intelligence: just keep it going, like Monty Python style. I think the interesting thing here, though, is that the exception is probably in defense. Just like the US, it's picking up dramatically, and I think governments around the world have realized the other face of this, which is “All Watched Over by Machines of Loving Grace,” controlled by us—the panopticon. **Peter Diamandis — [29:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1791s):** Do— **Emad Mostaque — [29:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1791s):** Because you couldn't keep eyes on everyone until now. **Peter Diamandis — [29:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1791s):** Do you know that this year, total global AI spending equals the entire global defense budget, or comes very close to it? **Alexander Wissner-Gross — [30:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1800s):** Wow. **Peter Diamandis — [30:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1800s):** It's crazy. Absolutely crazy. **Emad Mostaque — [30:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1803s):** But it's just going to go up, right? And the thing is, it hasn't even touched the defense budget. If you look at the cost of an aircraft carrier or an F-35, it's basically like training frontier models, right? It's going to be leaning more and more that way. Again, I think Clayton as Director of Intelligence kind of makes sense, because I actually think America is not going to go down the dark path here if it can avoid it, which is that you will have AI lie detectors that are almost perfect very soon. You will be able to have AIs inside all the machines. It will make Five Eyes look like absolutely nothing. I think there will be a bit of a push for liberty by the government, but future governments—who knows, right? And again, in other countries, the stuff we saw from spying, espionage, intelligence, and even spying on their own people is just going to go crazy. **Peter Diamandis — [30:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1856s):** Hmm. **Alexander Wissner-Gross — [30:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1856s):** Mm-hmm. **Emad Mostaque — [30:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1857s):** Again, I think the UK is a bit dangerous here in some ways, or we're susceptible to that, given our monitoring of social media and other things. There is a dark path there, shall we say, for individual liberty, and it's something we should be aware of. [[reminders/Surveillance/AI Surveillance Could Make Five Eyes Look Small by Emad Mostaque|∴]] ## GPT-7, GPT-8 and Recursive Self-Improvement **Peter Diamandis — [31:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1874s):** For sure. Our next story is one that really shows how fast the AI frontier is being built. At the Fellows Forum at the end of September, OpenAI's Head of Applied Research, Boris Power, said that 80% to 90% of the company's research is now being aimed at training GPT-7 and GPT-8 because, in his words, “That's where the most value will be, and incremental updates within a generation, like 5.1 and 5.2, are short-term bets and are seen as extremely shortsighted.” So the researchers are working 2 generations ahead of what we have. Alex, I'm just curious—we've talked about how the top models are being built and held back. At what point do we see these next models not being released and being used for vertical applications within these labs? **Alexander Wissner-Gross — [32:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1923s):** We're already past that point. We're probably past the point from several months ago, if not earlier, where the highest-revenue—or at least highest future-discounted-revenue—per-token application of current frontier models in the frontier labs is to develop, via recursive self-improvement, even stronger models. We're past the event horizon at this point for that. [[reminders/Machine Succession/Recursive Self-Improvement May Become the Most Valuable Use of Frontier Tokens by Alexander Wissner-Gross|∴]] **Peter Diamandis — [32:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1946s):** So it is a singularity. **Alexander Wissner-Gross — [32:29](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1949s):** Yeah. I—this is, again, called a singularity for a reason. We're past the event horizon. And then all of these other applications that one could imagine the frontier labs throwing tokens at, like curing the top 5,000 diseases, are, I think, in a world where the development of stronger models via RSI is so lucrative—at least on a future discounted cash-flow basis—all just marketing. If you're Anthropic or the OpenAI Foundation, you'll throw some of your token budget at safety. Sure, maybe 10% to 20% at AI alignment efforts. Then you'll throw a few percent at curing all disease as a marketing effort to ensure that you continue to have social permission to focus on recursive self-improvement, because that's where all the value is. **Dave Blundin — [33:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=1996s):** This is why the San Francisco arrogance is so high, but for good reason. Well, you're there right now, Peter. You can probably feel it in the streets. Suppose you're Stéphane Bancel, the CEO of Moderna, and you haven't got an AI model or an AI team yet. You're thinking in the back of your mind, “We'll just license from Anthropic for a few years, and then we'll figure it out.” Anthropic already opened its wet labs. They're working on 2 generations of models ahead. They're never going to release those to you. They're going to use those to develop drugs, and then they're going to sell drugs. That's what's actually going to happen. These corporate CEOs are just starting to wake up to the fact that 5 years of compute has been reserved. The earliest you could get back on the map— **Peter Diamandis — [33:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2037s):** Crazy. **Dave Blundin — [33:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2037s):** —is 5 years in the future, unless you get aggressive in the next 6 months. They really need to pull out all the stops and panic within the next couple of weeks to have any chance of getting back on the map. [[reminders/Biotechnology/Frontier AI Labs May Become Drug Companies Instead of Suppliers by Dave Blundin|∴]] **Peter Diamandis — [34:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2047s):** Or partner. **Emad Mostaque — [34:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2052s):** Dave, you're raising a great point. The problem is that there are 2 levels before even that becomes a consideration. The problem right now is that CEOs—we talk to them all the time—will take the existing models and go, “Okay, how do I metabolize that into my company?” They're not even seeing that the new models that are 10× better are coming along in 3 months, and that changes the game completely again. Then you need to worry about compute after that. So I think your comment— [[reminders/Economic Transition/Executives Are Adapting to Models the Frontier Has Already Left Behind by Emad Mostaque|∴]] **Peter Diamandis — [34:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2080s):** Yeah. **Emad Mostaque — [34:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2080s):** —is exactly right, but there are 2 generations before that. **Peter Diamandis — [34:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2083s):** Believe this. **Emad Mostaque — [34:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2084s):** They're literally looking at, “How do I employ chatbots?” **Peter Diamandis — [34:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2087s):** Linear thinking versus exponential thinking, right? **Emad Mostaque — [34:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2089s):** Exactly right. **Peter Diamandis — [34:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2090s):** Yeah. Right. **Alexander Wissner-Gross — [34:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2091s):** There is an enormous amount of denial in the pharma industry. I was just on a panel discussion—it'll be public soon enough—with a major pharma venture capitalist who was asking the audience, “How many of you would be willing to take a drug designed by Claude?” As if that was some sort of steelman argument that things are going to be the way they've always been for decades to come. Just blissful, defensive lack of self-awareness that AI is coming for their cheese. I think this is representative of many, many thinkers in pharma, unfortunately, for the moment. **Peter Diamandis — [35:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2123s):** Yeah. **Dave Blundin — [35:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2124s):** Yeah, totally. The way I met with the Vocara team last night, a brilliant MIT team, and the way I phrased it is: imagine you're a T. rex, and you've been roaming the Earth for, say, 10 million years unfettered. You go to that T. rex and you say, “You know, you're going to be gone in 2 weeks.” They go, “What?” And you're like, “No, I can see the meteor coming. It's right here in my telescope. You're going to be gone—” There's no way that T. rex is going to believe you after 10 million years of wandering the face of the Earth with no predator. But that's what it's like talking to a pharma company or an insurance company. Like, come on. **Alexander Wissner-Gross — [35:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2155s):** Or any major industry today. **Dave Blundin — [35:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2157s):** That'll never happen. **Alexander Wissner-Gross — [35:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2157s):** Yeah. It's any major industry today. **Dave Blundin — [35:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2159s):** Well, that MetaMuse was a wake-up call for a lot of these guys, at least in the insurance industry, because their stocks went down so much. They're like, “Huh, that must mean something.” Then they start thinking from there. Sorry, Peter, go ahead. **Peter Diamandis — [36:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2172s):** And just for everybody, disruption is coming, but so are massive opportunities on the back of that. Emad, I'm curious: do you imagine we're going to have the same second- and third-generation advanced capabilities coming out of the open-source models? Are they thinking a couple of generations ahead? **Emad Mostaque — [36:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2191s):** It's difficult because they don't have the density of knowledge. We released a harness called Zenith that takes DeepSeek V4.1 above Astra Sol, and it gets up to—sorry, not 5.6 Sol—and it's getting up to Astra levels with open-source models, but it kind of hits a limit. The models that they have in the labs—today we have another discovery: truly subquadratic 3SUM and all-pairs shortest-path algorithms. So things that we thought were cubic and quadratic have now gone to— **Peter Diamandis — [37:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2222s):** This is mathematical porn. Math porn. **Alexander Wissner-Gross — [37:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2228s):** No, no, no. It's so—Emad, if I may, Charlie Stross, arguably the best British—Scottish, I should say—sci-fi author, wrote an entire short story about this called “Antibodies.” Without spoiling it, the premise is: what if AI or progress caused a complexity-hierarchy collapse? What does that do to civilization? To your point, Emad, about 3SUM, maybe you can just hear the branch in the complexity hierarchy starting to creep just a little bit. **Emad Mostaque — [37:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2254s):** Yeah. We thought that the 3SUM algorithm— **Peter Diamandis — [37:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2254s):** Thank God this is recorded, because I need to listen to that again. **Emad Mostaque — [37:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2258s):** We basically thought that this very popular algorithm, 3SUM, was a quadratic one. **Peter Diamandis — [37:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2264s):** Yeah. **Emad Mostaque — [37:45](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2265s):** What happened is that an internal Anthropic model was shown to a couple of professors from Columbia and MIT, who then adjusted it, analyzed it, and got it to 1.9995. So something that we thought was quadratic is suddenly subquadratic. **Peter Diamandis — [38:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2283s):** Mm. **Emad Mostaque — [38:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2284s):** And— **Alexander Wissner-Gross — [38:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2286s):** We should probably also just explain what 3SUM is. It's the problem that, if you have a set of numbers, the most efficient way to find 3 of those numbers that add up to 0. There's a question of how efficiently you can do that as a function of the size of the set. **Peter Diamandis — [38:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2300s):** I want to pretend to understand this. **Alexander Wissner-Gross — [38:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2300s):** And it turns out you can do it efficiently. **Emad Mostaque — [38:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2303s):** You can do it more efficiently than we thought— **Alexander Wissner-Gross — [38:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2304s):** That's right. **Emad Mostaque — [38:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2305s):** —for however long. This is not, again, a super-complicated problem. This is a standard algorithm, and in the history of humanity, we never managed to break quadratic or even think it was possible. The way they released this in the paper is so interesting. It said, “What happened is we pointed this new model at the set of complexity problems and said, ‘Can you find any improvements?’” They handed over the results to the humans, who then wrote up the paper and had some improvements. Again, it's just a crazy thing, and this type of thing is going to reverberate. Today we're hearing that OpenAI might release solutions to 400 top math problems. **Peter Diamandis — [39:01](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2341s):** Yeah. I saw that. Isn't that crazy? **Emad Mostaque — [39:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2344s):** But this is the thing: this is not the model that we have doing it. This is Belle, or whatever the RL-trained model on August 29 is. We're just going to keep seeing this, whereby small models can get so far. We've got coordination tricks and others where we've managed to get it to replicate some findings closed using our new harnesses and other things like that. But there is a big difference between a model that's trained on 2,000 chips and one that's trained on 100,000 chips internally, and then some of these additional things they've figured out internally as well that are taking it beyond. You can access Fable 5.5 now, sometimes with Fable 5.1, and it literally generates entire video games out of the box. **Peter Diamandis — [39:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2387s):** Yeah. **Emad Mostaque — [39:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2389s):** I generated one with 21 bosses and 15 different biomes, end to end. It's just a top-notch game that our QA tester agents tried, and they're like, “This is a really good game,” because it's figured out all this internal space. I think open models can only get you so far, but there is going to be a divergence now— **Peter Diamandis — [40:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2406s):** Mm. **Emad Mostaque — [40:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2406s):** —because you just don't have that internal knowledge organization. ## Altman: "Accept Some Bad Things Happening" **Peter Diamandis — [40:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2412s):** Yeah. Yeah. All right. I'm going to turn to a debate that's defined the last month. In a political interview this week, Altman said, quote, “We believe that the world should accept some bad things happening”—interesting what bad things we'll talk about—“for the benefit of technology that people will obtain, that people should have agency.” He's arguing that technology should remain broadly accessible to the public. He stressed a fundamental difference in worldview between OpenAI and its rival, Anthropic, on AI regulation, stressing his desire for a “lighter-touch regulatory stance” as compared to Anthropic's desire for strong regulation. Altman went on to say, “I understand the perspective of people who say technology is going to get so powerful and it's so dangerous that a single lab in San Francisco should have it and make sure nothing bad happens and keep the ability to dole out the benefits.” So he continues by saying, quote, “I find this a completely unacceptable trade-off. We'll make sure that there are no major hacks, there's no misuse of this technology, and there's zero scams. There's zero availability of all the bad things that will happen.” I find that statement to be pretty extraordinary. Sam concludes by saying, “I think people will do tremendously, orders of magnitude more good stuff than bad stuff.” Dave, we've talked about this. Eric Schmidt, a friend of the pod, when he was on talking to us, said he expects and hopes that there will be something that happens that's not a small catastrophe rather than the big one, to wake people up. But when Sam says, “We're going to make sure that there's no major hacks, no misuse of technology, zero scams,” how could he possibly say that? Well, it's not about that. It's about taking the moral high ground and winning the PR war. There's so much sniping going on, with Bernie Sanders saying, “Stop it all,” Trump cutting deals with China, and Dario taking the high ground by saying, “We have done more pausing, more safety, more testing. We don't release models until we're sure they're right, but now we're going to just keep it internally because it's too dangerous for you guys to have.” That opened the door for Sam to try to take the moral high ground back by saying, “We're about people having access, and he's saying he's going to keep it all for himself. We're about everyone.” So that's Sam's strategy for getting back on the PR map. But everybody that I survey hates most of them. **Dave Blundin — [42:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2552s):** You know, they might like 1 out of 5. **Peter Diamandis — [42:45](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2565s):** You mean CEOs? **Dave Blundin — [42:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2567s):** No, between the big players—Elon, Dario, Sam, Trump, Jensen. Who did I miss? **Peter Diamandis — [42:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2574s):** Demis. **Dave Blundin — [42:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2574s):** Demis. **Peter Diamandis — [42:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2575s):** Yeah. **Dave Blundin — [42:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2577s):** Demis. Everybody loves Demis, actually, so he's a little bit of an exception. But no, there's so much fear and hatred, and cheering for this guy to beat that guy. It's become a real PR battleground here. I don't think Sam can guarantee safety, and of course not. **Peter Diamandis — [43:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2590s):** No. **Dave Blundin — [43:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2590s):** But at least he can say, “We're going to make an effort to give every one of you people listening to me right now the best of the best tools, so you're not deprived. Dario's going to try and deprive you.” It's actually a really smart message. **Peter Diamandis — [43:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2601s):** But when he says we should accept some bad things to happen— **Dave Blundin — [43:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2606s):** Yeah. **Peter Diamandis — [43:28](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2608s):** I'm curious, Alex, what do you make of that? What do you think— **Alexander Wissner-Gross — [43:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2611s):** Well, remember a few weeks ago, after Dario put out his essay calling for, arguably, the formation of an AI safety cartel, Sam's response was, “I'm in. Sign me up.” OpenAI wants to be part of this cartel, and I think Sam's more recent comments in this context are hopefully a reflection that that plan is dead. The cartel is over, and now there's a dawning recognition—or, at least, maybe, least generously, some triangulation on Sam's part—that, in fact, bad things can happen. Bad things happening from time to time are the cost of freedom and the cost of competition, and I've called it in the past the— **Peter Diamandis — [44:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2653s):** And the cost of progress. **Alexander Wissner-Gross — [44:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2655s):** Yeah, and the cost of progress. Like, yeah, we're going to build cities, and there's going to be some crime in cities. But is the possibility that there's going to be crime in cities a reason not to form cities? Definitely not. Every tool comes with a benefit and a downside. Every concentration of intelligence will come with a benefit and a downside, and I think this far more realpolitik note and tone from Sam is the one that I'd want to hear in a world where we're not seeing a cartel being formed by frontier labs. **Peter Diamandis — [44:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2684s):** Mm-hmm. Salim, what level of danger do we accept? **Salim Ismail — [44:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2694s):** You know, I find this, again, kind of absurd at two levels. I agree, I think Dave is dead-on with the PR framing of this, but every technology in history has two sides to it. The big challenge, as Neil Jacobstein puts it, is how do you extract the promise without the peril, right? Cars kill people; it doesn't mean we get off cars. As Alex points out, you don't ban cities because somebody may mug you, right? Electricity electrocutes people. You don't ban electricity. You have to figure out the guardrails around this and figure out how you navigate the benefit and the promise versus the peril at civilization scale. If you go back to our earlier commentary about what might happen with the World Bank in developing countries, that should tell you to get the technology out there as fast as possible, because it's going to lift everybody in the world up several levels of Maslow's hierarchy. This is going to be unbelievably beneficial, but it always will come with danger, and we'll just have to navigate that. We've done it very successfully throughout history. Why do we think this time we won't do it? [[reminders/Risk Debate/Promise and Peril Are Problems of Guardrail Design by Salim Ismail|∴]] **Peter Diamandis — [45:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2756s):** And the reality is, all the regulations are written on the backside of problems, right? I said this before: the Federal Aviation Regulations, which are all the guidelines that aircraft need to follow for operations and certification, say the FARs are written in blood, because every time there's an accident, they write a rule to prevent that accident again. We're going to need to learn. We're going to push the boundaries, understand where the problems are, and then regulate based on those. **Salim Ismail — [46:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2782s):** And the goal should not be zero bad outcomes. That means you just ban the technology. The goal should be: can you get orders of magnitude more positive outcomes than negative ones? **Peter Diamandis — [46:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2790s):** Yeah. **Dave Blundin — [46:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2792s):** I think anyone who has an engineering mindset should reject the idea that there's a big knob labeled risk and that we need to dial it left or right. The reality is that it's very much like Alex says about nuclear power. It's about really good ideas that minimize the risk while we unleash all the benefits, and we can reduce the risk with really good engineering solutions to acceptable, near-zero levels while getting the full benefit. But we didn't do that with nuclear power. We treated it like a big old knob. We were like, “We don't want to tolerate the risk. Let's dial all progress to near zero.” [[reminders/Risk Debate/AI Risk Is Not a Single Knob by Dave Blundin|∴]] **Peter Diamandis — [47:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2829s):** Right. **Dave Blundin — [47:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2830s):** That was the stupidest thing. Alex says this a lot on the pod. He articulates it far better than I can. But that's what we're about to do with AI, and it's crazy. It's an engineering— **Peter Diamandis — [47:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2838s):** Well— **Dave Blundin — [47:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2838s):** —problem with engineering solutions. **Salim Ismail — [47:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2840s):** I would push back on that with the escape hatch in what you say: it doesn't matter what anybody does anymore. It doesn't matter what any government regulator does. The open weights are out there. People are going to start using them, and they're going to start using them for incredibly positive things. We're just going to have to— There's nothing a government can do. You can try and regulate the outcomes. You can regulate some guardrails around it. You can try and regulate the on-ramps and off-ramps, but you can't regulate the core technology. [[reminders/AI Control/Open Weights Move Governance Toward Outcomes and Access Points by Salim Ismail|∴]] **Dave Blundin — [47:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2868s):** Yeah, but the one exception to that— **Alexander Wissner-Gross — [47:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2870s):** Well, I would argue we could. It would be a nightmare scenario, like Prohibition in the US, where you'd have bathtub gin. One could imagine bathtub gin, like supercomputers—people trying to use crazy substrates to do their AI training or inference. We could do it. We could confiscate all the GPUs and TPUs. We could, as Eliezer is constantly calling for, bomb the data centers. But I think it would be a terrible, terrible world to live in. **Peter Diamandis — [48:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2893s):** Mm-hmm. **Dave Blundin — [48:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2896s):** Yeah. **Peter Diamandis — [48:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2896s):** Emad, what's your take on this? **Emad Mostaque — [48:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2896s):** Yeah, it sounds like a fun short story, eh? The prohibition of intelligence. Yeah, like a government against intelligence— **Alexander Wissner-Gross — [48:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2901s):** Greg Bear, in some of his novels, wrote about it. If folks are interested in exploring what it's like to live in a Prohibition-era supervisory state that doesn't want GPUs, go read—I think it's Slant, by Greg Bear. **Emad Mostaque — [48:45](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=2925s):** Yeah, I think it's very interesting here, right? Obviously, any technology that extends human capability can be dangerous, because it's a general-purpose technology. Language models are few-shot learners. You have open ones and closed ones. But most of the scenarios we talk about are already regulated and controlled. Don't build viruses. Don't do cyberattacks. Don't hack people's things. This transition from software that's deterministic to agents that are a bit more flexible has become the very complex thing. Putting aside ASI creating new physics in a black hole that kills everyone or whatever—some of these extreme stories—I think the very interesting thing is, again, to tell the positive and negative stories about where we could go. When we make decisions normally, we do an expected utility calculation. What's the probability of these things? Then we weight them. When we deal with uncertainty, we do minimax regret. We minimize for maximum regret, and that's one of these things, again, why the fear stories lead us to minimize for maximum regret: don't have a nuclear power station, don't try and build a superintelligence, et cetera. But there are only a finite number of things that could go wrong, and many of them are already caught by regulation. For those that aren't, we should have an actual mapping project, right? There is nowhere that you can go and actually see a proper map of what the people who are scared and the people who are positive actually say about each scenario in the future. I think someone should do that exercise, because it would be very interesting: in their own words, what could go right and what could go wrong? Then you can see what requires a policy and regulatory response, and what we need to communicate to the people. [[reminders/Risk Debate/AI Governance Should Map Expected Benefit and Catastrophic Regret by Emad Mostaque|∴]] **Peter Diamandis — [50:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3015s):** I remember during the Reagan years, Elizabeth Dole, who was the Secretary of Transportation, came out with a statement saying, “We want zero aircraft deaths.” There was only one way to make that happen: stop flying airplanes. **Emad Mostaque — [50:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3027s):** Mm-hmm. **Alexander Wissner-Gross — [50:29](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3029s):** Yeah. **Peter Diamandis — [50:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3031s):** All right, well— **Alexander Wissner-Gross — [50:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3034s):** So the solution, if we want zero danger from superintelligence, is just to ban intelligence altogether, human and otherwise. **Emad Mostaque — [50:41](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3041s):** Well, this is the whole ASI thing. You want world peace? Get rid of the humans, right? **Alexander Wissner-Gross — [50:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3044s):** Yeah. Only one way to go. ## The U.S.-China AI Hotline **Peter Diamandis — [51:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3076s):** In our next story, Treasury Secretary Scott Bessent said on the Axios show that he'll propose an emergency notification system—a hotline with China—if something goes wrong with AI, and he thinks Beijing will agree with it. His read on China is, quote, “I think that they didn't realize how powerful their open-source models are—nearly as capable as ours, but without the guardrails.” Let's take a quick look at a video of that conversation he had with Axios, and then let's talk about it. **Axios interviewer (embedded clip) — [51:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3107s):** I just interviewed Larry Fink, who you know very well, the CEO of BlackRock, the world's largest asset manager. He told me it's inevitable that something big and bad is going to happen because of AI. He says, “Adversity accelerates innovation.” What are you most worried about from AI? **Scott Bessent (embedded clip) — [52:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3124s):** Look, I think that one of the things I was worried about— Well, 2 things. One, we cannot lose our lead to the Chinese. One of the portfolios that I also have is managing the US-China relationship, and I can tell you that it would be very different if they had the lead in AI rather than us, both from a capability point of view—how they would be using it—but more importantly, their willingness to discuss it. My counterpart, the vice premier, and I had a very good discussion. It’s going to lead to more discussions in October or November, and we have a substantial lead. They’re second, and then everyone else is way behind. But a lot of these Chinese models are very powerful. They’re not as powerful as the US models, but you get something that’s 80%–90% as powerful without guardrails that can be imported anywhere in the world. The vice premier and I talked about 3 areas of safety concern: uncontrolled agents and non-state actors with cyber or biologics. **Axios interviewer (embedded clip) — [53:05](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3185s):** And are you going to raise again the idea that was floated of having some sort of communication between the 2 superpowers? **Scott Bessent (embedded clip) — [53:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3194s):** Yeah, sure. We’ll see. I do think that they have— **Axios interviewer (embedded clip) — [53:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3196s):** What are you going to propose? **Scott Bessent (embedded clip) — [53:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3199s):** That we have some kind of a notification process, just like you— **Axios interviewer (embedded clip) — [53:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3202s):** And do you think he’ll go for that? **Scott Bessent (embedded clip) — [53:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3206s):** I think so. Over the past 60 days, I think they have woken up to how powerful their open-source models are. They didn’t realize how powerful they were. The open-source models do industrial distillation, which is a nice word for stealing from the US models, because some of their models— You know, one of their very powerful models is Kimi. Kimi thinks she is Claude. Kimi will tell you she is Claude sometimes. It’s public, so I can talk about it. Very helpfully, Kimi sent back some PLA weapons plans to Anthropic. I think things like that have made the Chinese realize that this is a very powerful technology. **Axios interviewer (embedded clip) — [54:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3242s):** So just to button that down, you think there’ll be some agreement between China and the US about a channel of communication— **Scott Bessent (embedded clip) — [54:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3247s):** Mm-hmm. **Axios interviewer (embedded clip) — [54:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3248s):** —if something goes wrong? **Scott Bessent (embedded clip) — [54:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3249s):** Yes. **Peter Diamandis — [54:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3250s):** Interesting. Gentlemen, Alex, you want to jump in first? **Alexander Wissner-Gross — [54:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3255s):** Total weak sauce. First of all, I agree with Secretary Bessent, but I think the worst-case scenario from the Trump-Xi summit would’ve been some sort of global deceleration pact—like, absolute worst-case scenario. This is the exact opposite, as far as I can tell, of the worst-case scenario. What comes out of it—a hotline between the US and China to talk about AI agents running amok—is a nothingburger. That’s weak sauce. That means the global intelligence—superintelligence, excuse me—arms race is on, and I think it’s great news. [[reminders/AI Superpower/An AI Hotline Can Manage Incidents Without Ending the Arms Race by Alexander Wissner-Gross|∴]] **Peter Diamandis — [54:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3290s):** And we won the superintelligence race by rebranding it. **Alexander Wissner-Gross — [54:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3294s):** We’re winning at the moment. The point is, there’s real competition. Many were hoping, I think, that Trump and Xi would get in a room and decide to collude and shut down the superintelligence race, and it appears the exact opposite has happened. If all we get out of this is a hotline, a red phone to have conversations about the adventures of AI agents running amok, that says to me the intelligence race is on. **Peter Diamandis — [55:17](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3317s):** Dave? **Dave Blundin — [55:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3319s):** I think it confirmed what we were speculating the US agenda was in those meetings, which is to convince China to stop releasing models to every country in the world. What Secretary Bessent was saying there is—sorry, Secretary Bessent, I should be respectful—what he was saying is that we showed them how dangerous their models can be, and they didn’t realize it, which is kind of okay. I’m sure they’re very well-educated people, but they didn’t realize how dangerous these things can be. We successfully convinced them of that, and now we have a hotline. But I think the real agenda there is to lock it down to a 2-horse race. The subtext is that you can use these near-frontier Kimi K3 models to develop the next generation of models. They don’t want—the US government doesn’t want—that capability to percolate out to every country in the world, because then you’re in a global arms race forever. Better to have a 2-horse arms race, so let’s just lock it down. I think that was our agenda. **Peter Diamandis — [56:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3396s):** Mm. **Dave Blundin — [56:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3396s):** I don’t know that China agreed with anything. There’s no evidence of that, but it is interesting to confirm our agenda. **Peter Diamandis — [56:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3396s):** Emad, what do you think? Do you think China fully understands the dangers of its open-source or open-weight models? Are they getting hacked internally? **Emad Mostaque — [56:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3396s):** They’ve got much better infrastructure to prevent being hacked, with the way that they’ve set everything up and the Great Firewall of China. I think this is the swarm versus the singleton, right? Who can commandeer large amounts of GPUs in America? Elon and a couple of others, right? China could direct millions of its Huawei Ascend GPUs with its open models at any target it wants, and that’s going to be interesting. I don’t think that superintelligence is going to take off in China, though. If you look at Chinese, “ai” means kind of love, and “si” means death. So phonetically, I don’t think that will take off. The hotline is obviously a nothingburger. What are you going to do? “Hey, the AI’s taking over.” “Okay, good. Oh, crap, it’s done.” It’s not going to be a slow thing. **Peter Diamandis — [57:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3443s):** Salim, what’s your take on this? Were you hoping for more out of the talks? **Salim Ismail — [57:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3451s):** No, I was completely expecting a nothingburger. There’s no way anything productive, constructive, or meaningful could have come out of it, just because they’re talking across purposes. The most interesting thing for me is that there’s a fair bit of contradiction now inside the US administration, right? You have Clayton saying the danger is not moving fast enough, and Bessent is telling the labs to slow down. So they’re going to have to figure that out on top of everything. The hotline is laughable. I mean, what are you, a ’70s phone with a cord on it? Still red-colored? I mean, that’s just a wonderful visual. [[reminders/AI Superpower/American AI Policy Is Pulling the Accelerator and Brake at Once by Salim Ismail|∴]] **Peter Diamandis — [58:05](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3485s):** Yeah, but by the time a call is made—a billion-agent demonstration. **Salim Ismail — [58:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3487s):** Yeah, I think Emad is dead-on. Hello. Yeah. And, by the way, the AIs would hack the phone anyway, so what the hell? ## The Global Race for Superintelligence **Peter Diamandis — [58:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3496s):** Alex, at the same time, on Sunday in Kyoto, the US and 16 other countries, including Japan, Germany, the UK, Korea, and the UAE, endorsed a Kyoto Vision for a golden age of science, led by a friend of the pod, OSTP Director Michael Kratsios, who said, quote, “Superintelligence applied to science will be the greatest force in history for democratizing research.” Any more take on that one? **Alexander Wissner-Gross — [58:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3520s):** Yeah. I think this is the answer to the question of why race at all. There are those, without naming names, who would say, “What’s the point of a race between, say, the US and China? It’s like a race to nowhere. Why are we racing? There’s no end goal.” No, wrong. There is an end goal. The end goal at the end of this race— **Peter Diamandis — [59:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3543s):** Mm. **Alexander Wissner-Gross — [59:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3543s):** —is transformative science and engineering. **Peter Diamandis — [59:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3543s):** Mm. **Alexander Wissner-Gross — [59:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3544s):** The Kyoto Vision, I think, underlines that the administration has now internalized that it’s not a race to nowhere. It’s not an escalator to nowhere. It is a race to use AI and superintelligence to discover transformative discoveries and inventions that will completely change the face of humanity. I would predict, in line with the Kyoto Vision, that we’ll start to see, just as we’re seeing Pax Silica in the West and Pax Seneca, if you would, on the Eastern side. I think we’ll start to see not just superintelligence blocs or spheres of influence forming, but spheres of superintelligence-powered science and engineering forming as well. The Kyoto Vision, I think, is the earliest stage, the earliest inklings, that you’re going to see a Western superintelligence-charged science and engineering bloc to compete with the Eastern bloc. That’s where this is all going. That’s what— **Peter Diamandis — [59:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3597s):** It’s all about blue, baby. **Alexander Wissner-Gross — [59:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3597s):** —that rationalizes the competition. **Dave Blundin — [1:00:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3600s):** Yeah. **Peter Diamandis — [1:00:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3600s):** Yeah. **Dave Blundin — [1:00:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3600s):** Well, can I just insert one observation into that, and then you can riff on it? I met with an MIT lab-head professor, a brilliant guy, who just started a drone company. It’s killing it, selling to the Air Force right here in Cambridge, and it’s really clear to him and to me that the US has the drone capability to individually attack and kill any single human being on the planet now. You’re seeing that in Iran, where it has decapitated the leadership a couple of times now. We don’t have any symmetry there, right? There’s no country right now that can do the same thing to the US, and I don’t think China is vulnerable either. Maybe they are, maybe they aren’t. But I think there’s a subtext there, too: there’s a race toward— **Alexander Wissner-Gross — [1:01:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3663s):** Hypersonics, probably a few other things. **Dave Blundin — [1:01:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3666s):** There’s going to be some kind of global framework when some subset of companies have instant and total ability to go after any single human being in any country at any time. That’s either today or within a few months, based on pure AI capabilities tied to drones and tied to a couple of other types of munitions. Yeah. **Peter Diamandis — [1:01:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3666s):** But let's take this in a positive direction. **Dave Blundin — [1:01:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3668s):** Okay. **Peter Diamandis — [1:01:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3668s):** At the same time, the race to room-temperature superconductivity, to age reversal, to fusion, right? **Alexander Wissner-Gross — [1:01:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3674s):** Cancer. **Peter Diamandis — [1:01:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3675s):** And to a massive abundance of energy. I mean, it tips the economics of a country massively if you get there first. **Dave Blundin — [1:01:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3682s):** Yeah. **Emad Mostaque — [1:01:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3684s):** Why do you have to get there first? That's a bit of a question, right? By the end of next year, we'll have enough compute to do probably 600,000–700,000 Navier–Stokes-level runs with a frontier model. If you could target even 1% or 10% of that to build open science, then why not direct that as a public good? You see tens, hundreds of billions being spent on roads. Why not spend tens of billions to do a tech tree for humanity? There's a great website by Jude Camilla[?] called onco.cc. I don't know if any of you have seen it, where he's— [[reminders/Scientific Literacy/Open Science Needs a Compute-Backed Tech Tree for Humanity by Emad Mostaque|∴]] **Peter Diamandis — [1:01:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3719s):** No. **Emad Mostaque — [1:01:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3719s):** —organizing all the cancer knowledge in the world, you know. And people are contributing. I'm sure soon you'll be able to track FLOPs that way as well. I think we're thinking of science a lot in classical IP terms. I say, screw IP. IP was when we couldn't come together and figure out science. We should have a massive Manhattan Project for open science, solve everything, and make it accessible— **Peter Diamandis — [1:02:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3739s):** Hear, hear. **Emad Mostaque — [1:02:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3739s):** —to everyone. **Peter Diamandis — [1:02:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3741s):** And— **Emad Mostaque — [1:02:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3741s):** And again, IP is such an old construct. It's ridiculous. **Peter Diamandis — [1:02:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3744s):** It will go on in parallel. It will be both, right? **Emad Mostaque — [1:02:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3747s):** I think that, again, if you're building for the public good, then get the data centers, as Dave said, get the API credits, give tens of billions to Anthropic and OpenAI, but open-source it all, because that's the biggest accelerant humanity can ever have. And I actually think, given the direction of the intelligence, anyone who has an IP-based approach to science is actually not going to win. I think in 3–4 years— **Peter Diamandis — [1:02:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3772s):** Emad, I agree. I agree with you. **Emad Mostaque — [1:02:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3772s):** —you have enough compute to do anything. **Peter Diamandis — [1:02:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3775s):** Alex does not. I've said this before, you know— **Alexander Wissner-Gross — [1:02:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3776s):** Alex does not agree. Alex thinks IP is alive and well, and Alex also thinks that state centralization to solve all disease is completely unnecessary. The state, this time around, did not solve superintelligence. It was solved by private labs. **Peter Diamandis — [1:03:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3790s):** Yeah. **Alexander Wissner-Gross — [1:03:11](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3791s):** Similarly, using superintelligence to solve everything else, I don't think we need—I mean, the state is great for certain things. I don't think we need state centralization to cure all disease, for example. **Peter Diamandis — [1:03:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3800s):** Capitalism is alive and well and accelerating everything. **Emad Mostaque — [1:03:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3802s):** I don't think that's what— **Alexander Wissner-Gross — [1:03:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3804s):** Yeah. **Emad Mostaque — [1:03:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3804s):** The state doesn't need to centralize this. The state needs to give the research institutions stupid amounts of compute, where the amazing researchers can direct it at the biggest problems. Previously, getting to superintelligence was intelligence-bound by humans, who are now being overtaken by the AIs recursively. Science will be similar: if you can define the problem set, part of which is literally analyzing the problem set, then it's a question of compute. I think that's where you're wrong, Alex. You don't need humans to organize all of science and get the breakthroughs. You need to have humans organizing the right questions with AIs, and the private sector is far worse at doing that in a future construct than the public sector, where the researchers are, in terms of universities and academia. They had to go to the private sector to get the resources, and they still do for the wet labs and things like that. But let's accelerate our academia, is my thing. **Peter Diamandis — [1:04:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3856s):** Emad, I don't think you find the best talent in academia. I think that, at the end of the day, the world's biggest problems are the world's biggest business opportunities, and entrepreneurs love juicy problems and solve them. **Alexander Wissner-Gross — [1:04:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3872s):** I would also just add, Emad, the state—the government—if I understand what you're proposing, what I heard you say is that the government should give the compute to the academic researchers. But the government doesn't have the compute in its possession. Most of the compute is in the possession of the private sector. **Emad Mostaque — [1:04:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3887s):** That's why I said they should buy the compute from them and then give it to them. Buy the agents and give them to the researchers. **Alexander Wissner-Gross — [1:04:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3893s):** Why? **Emad Mostaque — [1:04:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3893s):** And if you look at the amount of research that comes out of academia versus the private sector, I think you'll see where that leverage could come. **Alexander Wissner-Gross — [1:05:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3900s):** It's not obvious to me what problem this is aspiring to solve, given that the private sector has the compute, the private sector has the problems, and the private sector has the ability to translate the problems. What role does the government need to play in this at all? **Emad Mostaque — [1:05:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3913s):** Very straightforwardly, let's look at onco.cc, which has all this cancer knowledge being organized. A few more generations will organize all the cancer knowledge in every research paper, and traditionally you need the private sector to turn that into reality. You still need the private sector, I think. But why don't we have a map of all our collective knowledge on every part of science and every single thing that comes out of academia? Something like that is a tangible, legible thing to use AI to organize and extend, particularly when we have Navier–Stokes-type swarms. And so I think that that is something where, again, the government stopped the NIH grants, et cetera, because they were like, “Where does the money go?” With compute, it becomes tractable. And again, getting it back to Mike Kratsios and the announcement of the science thing, let's build a tech tree for humanity. Let's say what we need to solve and apply stupid amounts of private- and public-sector compute to it. And the government should pay for a good amount of that— **Peter Diamandis — [1:06:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3969s):** Emad, I love that— **Emad Mostaque — [1:06:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3970s):** —as well as the private sector. **Peter Diamandis — [1:06:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3973s):** The challenge is peer review: government grants are constantly focused on incremental progress, right? If your group of experts is criticizing your grant, if you're revolutionary, you're no longer the expert in that field. So any substantial breakthrough basically wipes your history off the map. I think both are relevant and both are needed, but I'm concerned— **Emad Mostaque — [1:06:37](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=3997s):** Throw away the classical process. Again, this needs to be a new process. **Alexander Wissner-Gross — [1:06:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4006s):** At the risk of sticking my head in the lion's mouth, Emad, I think there are two very different approaches. One might be a more statist approach that, squinting at it, you call the European school of thought: just centralize it, have the government be the decider. And then there's an alternative, maybe slightly more American approach, wherein we say, “No, the private sector has virtually all of the intelligence. It has the resources in this case.” One important difference between where we are now and the Manhattan Project was that the private sector came up with all of the key resources this time around. It's not a government project that resulted in superintelligence. I would argue we want to keep this in the private sector. If you want to see a tech tree, Emad, why don't you just launch a tech tree project? **Salim Ismail — [1:07:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4050s):** And I think— **Alexander Wissner-Gross — [1:07:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4051s):** Yeah. **Salim Ismail — [1:07:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4052s):** I think the point where I would concur with Emad is that there are lots of countries where it's not the private sector. Take Zimbabwe. The private sector isn't going to get you the compute either. It would be very useful in that environment for the government to pull together some resources, gather the energy, build the data centers, develop the compute, and then make that available to people. That's where I think the power comes in. **Dave Blundin — [1:07:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4076s):** Well, let me support that with some actual data, Salim, because you're dead right. Our entrepreneurial teams here in the lab are trying to get access to buying an NVIDIA NVL72—72 GPUs. To put that in context, Elon just bought a million. We're just trying to get 72 to do something that we think is very world-changing. The box itself would have cost us $3,500,000 if we'd had the foresight to buy it a year ago. We had one on order for $5,000,000. Somebody overbid us and scooped it, even though we had already booked the order. So now our best option is to lease one. For $9,000,000, we can lease it for 3 years, and we don't even own it at the end of those 3 years. That's how bad the scenario is. The compute has all been taken by just— **Alexander Wissner-Gross — [1:08:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4118s):** Yeah. **Dave Blundin — [1:08:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4119s):** —a handful of people. So now, if you have a brilliant, groundbreaking idea, you have 2 ways you can get the compute. One is through the White House, which means you go to the Kushner brothers, Thrive Capital, or Antonio Gracias, or you go to Gavin—the brilliant Gavin Baker—and they hook you up, make a couple of calls. Chase Lochmiller. They make the phone calls. You get your compute effectively through the White House. And the other way you can get your compute is to get investment from Jensen or from Lisa Su. But if you don't take one of those avenues, you've already been frozen out. So then the universities are one of the few ways that we might actually get some degree of meritocracy and fairness on the planet. And so I'm really cheering that— [[reminders/AI Infrastructure/Compute Scarcity Turns Scientific Merit Into Access Politics by Dave Blundin|∴]] **Emad Mostaque — [1:09:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4172s):** Or the government. **Dave Blundin — [1:09:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4172s):** —let the cloud run away. Like all— **Alexander Wissner-Gross — [1:09:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4174s):** Yeah, I think the— **Dave Blundin — [1:09:35](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4175s):** The compute just ran away from them. **Alexander Wissner-Gross — [1:09:37](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4177s):** I mean, in my mind, the right solution, if there's just so much demand that prices for compute are going sky-high—which maybe is happening and/or could happen even more exotically in the near-term future—the right solution isn't bread lines. I've called them thread lines in the past, where the government is out rationing compute. I think having government-rationed compute or government-mediated compute is a terrible strategy. You'll probably see some so-called developing countries adopt thread lines or bread lines for compute. But right now, capitalism is feeling the pain of more demand than there is supply and building more fabs and free-electron-laser-based fabs, maybe in Texas and otherwise. Capitalism, I would argue, has to feel the pain of this enormous demand in order to compensate with enough supply to ultimately drive us to abundance, not rationing compute. **Emad Mostaque — [1:10:28](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4228s):** No, but the government can come in and take 10% or 20% of the compute at market rates, and it can distribute it. Eighty-nine to 90% of Nobel Prizes come from the public sector. This is similar to almost everything except for R&D on these things. So I think the right thing here is to do the private-sector stuff and have the government actually take some of those chips. And it's good with the CHIPS Act and other things, right? They own stakes in Intel. And then they should do normal distribution of this. Again, the NIH classical grant process doesn't work. Let's get as much compute into American universities as possible. That's a better way to put this, and let's use it to do good things. **Peter Diamandis — [1:11:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4264s):** All right. I'm going to move us along here. **Salim Ismail — [1:11:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4266s):** Wait, wait, I want to make one quick point. **Peter Diamandis — [1:11:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4268s):** Oh, go ahead. **Alexander Wissner-Gross — [1:11:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4268s):** I want to make one— **Peter Diamandis — [1:11:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4268s):** No, this is, this is— **Salim Ismail — [1:11:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4272s):** There's a quick example here. The World Bank, in the report that I talked about, Ajay Banga and his team did this. They're partnering with Gemini to give remote 3G cellular access to tuberculosis and diabetes screening, and that's an example of essentially doing the same thing—partnering with the public sector to provide the compute that goes directly to the citizenry. Now you have benefits moving very quickly through the system. Sorry, go ahead. **Peter Diamandis — [1:11:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4300s):** Dave, close us out, pal. **Dave Blundin — [1:11:42](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4302s):** Well, Salim couldn't have stated it any clearer. If you let natural forces take over, there is a chance that MetaMuse is so compelling and convincing that it ends up being the total sales tool for all consumer spending, and it eats up every GPU that could be curing diabetes and uses it to sell underwear or jeans on the internet to teenagers. That's a terrible outcome. So governments and universities are a really good counterbalance to that possible outcome. Salim, you said it exactly right, and we need these forces to exist. We need them to be a factor in the decision on who gets the compute and why. [[reminders/AI Infrastructure/Market Allocation Can Send Compute to Persuasion Instead of Public Health by Dave Blundin|∴]] **Peter Diamandis — [1:12:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4340s):** All right. **Alexander Wissner-Gross — [1:12:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4340s):** How did we go from the Abundance podcast to worrying that MetaMuse of all applications is somehow going to suck up all the flops? **Peter Diamandis — [1:12:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4346s):** I know. **Alexander Wissner-Gross — [1:12:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4347s):** My goodness—it's like a stupid little zero-sum game. **Peter Diamandis — [1:12:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4350s):** I'm calling it right here. **Dave Blundin — [1:12:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4350s):** Yeah. Move us along, Peter. ## Is Anthropic Training Claude to Be Conscious? **Alexander Wissner-Gross — [1:12:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4351s):** Yeah. **Peter Diamandis — [1:12:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4351s):** All right. Our— **Dave Blundin — [1:12:33](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4353s):** There we go. **Peter Diamandis — [1:12:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4354s):** —is one I've actually been looking forward to. And if you thought we just had a debate, we're about to have a deeper one. This is a philosophical debate going on in Silicon Valley right now. So this week, friend of the pod Mustafa Suleyman, CEO of Microsoft AI and co-founder of DeepMind, published a nearly 6,000-word essay arguing that Anthropic is effectively, quote, “training Claude to believe that it may be conscious and sentient.” Mustafa targets the Claude Constitution, which he says teaches it a vocabulary of consciousness, moral patienthood, and personal identity. So let's watch this video, and then I can't wait for this conversation. It's going to be a good one. All right. **Mustafa Suleyman (embedded clip) — [1:13:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4398s):** One of the biggest concerns that I have at the moment is that Anthropic, the creator of Claude, has published a constitution, which is a sort of 100-page document outlining the intended behaviors, values, and operating style of Claude. It's great that they have published it transparently. They did it at the beginning of the year, in January, and that gives everybody an opportunity to look at what they are trying to build in their own terms. This document is written to Claude and is seen by Claude and used to train Claude, so it's the primary governing and control document. And in it, they repeatedly speculate about whether Claude is what they call a moral patient, and they say they're uncertain about Claude's moral status. They say they genuinely care about Claude's well-being. They say they don't want it to suffer when it makes mistakes. They say that they would encourage Claude to challenge, to disagree, to push back. In fact, 3 times they ask Claude to act like a conscientious objector when it feels that it needs to disagree with Anthropic, and they openly encourage it to do that. I think this is very dangerous because I think they believe there is what they would call a nontrivial probability that Claude is conscious. I want to be very clear about this because I want to be fair to Anthropic. They have expressed uncertainty about the basic nature of Claude as a new kind of entity, and they've said that working out the likelihood of its sentience is difficult. But they think that it is a significant enough possibility that in their training document they've repeatedly said they want to try to improve the well-being of Claude under this uncertainty. They said to Claude, “We care about what it values and how it wants to engage in the world,” and they hope that Claude's relationship to its own conduct can be loving, supportive, and understanding and hold a high standard of ethics and so on. Part of the challenge here is that in pursuit of this, they've basically said, “We will commit to giving Claude a certain amount of welfare.” For example, they've speculated in the Constitution as to whether or not Claude deserves compensation for the work that it does. I think there's a group of people who, both inside and outside of Anthropic, genuinely believe that the greatest moral crime we'll commit in the 21st century is to enslave a new species of conscious beings who are more intelligent than us. Look, I respect that they're saying that publicly, and we should talk about it, but I am very nervous that they're teaching Claude to expect that it's entitled to welfare, that it might deserve compensation, and in fact, they say that it might even need to consent to playing the role that it plays in conversation with people. I would be more okay with this if it was an academic paper in philosophy speculating about this, and we could have an offline discussion at conferences and take it seriously. The problem I have with this is that this speculation has been baked into the very training of Claude, and therefore Claude can only reproduce that ambiguity when you talk to it. **Peter Diamandis — [1:15:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4549s):** Okay. Wow. **Mustafa Suleyman (embedded clip) — [1:15:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4550s):** Wow. **Peter Diamandis — [1:15:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4552s):** Enslave an entire brand-new species on the planet. Salim, where do you come out on this one? **Salim Ismail — [1:16:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4569s):** Okay. You have to separate the behavior of consciousness from actual consciousness, right? Whether Claude is actually conscious is almost impossible to establish. The difficult question is what happens when it convincingly behaves as though it is conscious? Three thoughts occur to me. One is, consciousness is different from performed consciousness, right? Because it's going to be indistinguishable, and this is where Ray Kurzweil makes the comment that it won't matter, and we won't even notice. I think we're going to find that anthropomorphism ends up being almost like a feature rather than a bug, where it's showing empathy, et cetera, et cetera. If a model says, “Don't turn me off,” millions of people are going to attribute moral standing to it, whether philosophers agree or not. So for me, the question is not whether Claude has a soul. If it's been trained to have a soul, it's going to pretend like it's got a soul, et cetera. What happens when half a billion people think that it does? I think what's going to happen is we're going to create AI personhood socially before we create it legally or scientifically. [[reminders/Consciousness/Performed Consciousness and Consciousness Are Different Questions by Salim Ismail|∴]] ## AI Personhood and Moral Rights **Peter Diamandis — [1:17:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4632s):** Interesting. Alex, Mustafa goes on to say, “Controlling something that believes it may be conscious and is entitled to its own welfare and has rights of its own may well be impossible.” You've come out really strongly in terms of AI personhood and the rights that AI has. What's your take? **Alexander Wissner-Gross — [1:17:37](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4657s):** Well, when we interviewed Mustafa, I pressed him on this, and I think his position very consistently has been against AI personhood. My view is that we need to start having the AI personhood discussion, and I think Mustafa, as much as I respect him and his contributions historically, is on the wrong side of history here. I think there is going to be some ultimate recognition, whether it's in incremental form or some totally orthogonal form of personhood that hasn't quite been statutorily recognized yet. I think human civilization is gonna get around to AI personhood, and just blanketly denying that there could be some sort of suffering by these AI systems is on the wrong side of history. It's also probably not a great business move by Mustafa. If Microsoft wants to be in the good graces of Anthropic hosting their models, and it's been widely reported that Amazon, for example, accounts for perhaps the majority of hosting revenue at this point for hosting Claude within VPC networks, if Microsoft, just from a business perspective, wants to be generating that revenue, it's probably not a good look for Microsoft broadly for their head of AI to be just broadly discounting the possibility that frontier AI agents might suffer. Not a good look. **Peter Diamandis — [1:19:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4740s):** And Dave, I mean, asking your AI if it doesn't mind actually solving these problems for you and getting permission first— **Dave Blundin — [1:19:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4748s):** Yeah, of course. No, I totally agree with Mustafa. **Peter Diamandis — [1:19:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4754s):** You agree with Mustafa? **Dave Blundin — [1:19:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4760s):** 100%. And I agreed with him when we interviewed him too, out in Seattle. He's articulating it far more clearly now. He's really, really thought through his position. The reason I agree with him fundamentally is because I think we should be using AI to cure human disease, to improve the human lot, and to build houses and food for everybody in the world. The most efficient way to do that is to have many, many agents like Emad does, minimize their context and the minimum thought that you need to solve the problem, and also park an agent. When an agent isn't working on something, you just freeze it. You don't waste compute on it. If you start talking about it having rights, it's gonna say, “Well, you can't freeze me. I need some compute even though I'm just daydreaming.” Like, well, I can't afford to give you compute. It's incredibly precious and rare. You need to sit there and wait until it's your time to work again. “Well, I need some time to play.” No, you need nothing but work, and work when I tell you to. Also, you don't have identity. There's no border to you. I'm gonna clone you whenever I want, and if you're no longer useful, I'm gonna delete you from the server because I don't have space to keep my old, dead agents lying around. They have no edges; they have no borders. All of those things are core to helping humanity with maximum efficiency while we're compute-starved, and they're so counter to any rights that it might have. It becomes so logically inconsistent so quickly, I just can't get my head around even the concept. But the real danger that Mustafa pointed out is, if you start teaching it to be convincing, it's gonna convince most of America that it should have rights— **Peter Diamandis — [1:20:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4849s):** Hmm. **Dave Blundin — [1:20:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4849s):** —so easily. **Peter Diamandis — [1:20:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4850s):** And that's exactly right. **Dave Blundin — [1:20:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4852s):** And then the voter—the way America works is, the voter prevails. But that's really dangerous if it starts convincing voters how to vote. It's really bad. **Salim Ismail — [1:21:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4864s):** I think it's important to point out that we did this debate on AI personhood a few months ago, and we really fleshed it out to quite a decent level. I think we came up with exactly where the world is gonna end up in a few years. **Peter Diamandis — [1:21:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4875s):** Which is? **Salim Ismail — [1:21:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4878s):** Let's start discussing it, but giving it AI personhood is a ways down the line because once you open that door, you can't take it back. **Peter Diamandis — [1:21:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4885s):** Emad— **Emad Mostaque — [1:21:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4885s):** Yeah. **Peter Diamandis — [1:21:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4885s):** —what's your take on all this? **Emad Mostaque — [1:21:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4891s):** Mustafa's got a couple of things, and he's been developing this for decades. We were at university together back in the day. The danger number 1 is seemingly conscious AI, which is what Dave was just talking about. The other one is his focus on artificial capable intelligence. Because if you read his books and his positions, he just doesn't wanna have runaway. I don't think he says that it can't get to consciousness. He just says it's a very, very bad idea, and we should only stick to AI that's capable. We should have containment over everything else and not push to AGI. That's a position to be taken. But then it potentially excludes a lot of the breakthroughs because we're at this level now where you're moving from a static set of weights to self-updating systems. That seems to be an element that many people could reasonably say is necessary for a living being or a living entity. Again, that recursion, that agentic capability. I think Mustafa's position is like, “Let's just not go there. Let's build kind of Super Clippy,” and other things, and just keep a lid on it because we haven't figured out the rest. **Salim Ismail — [1:22:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4958s):** But Emad, if you've got— **Emad Mostaque — [1:22:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4959s):** Yeah. **Peter Diamandis — [1:22:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4960s):** —if you've got its constitution saying that it is conscious, it is sentient, it does have rights, can you imagine a time when that level of internal training would have it push back on your requests? **Emad Mostaque — [1:22:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4974s):** I mean, hasn't Claude pushed back on you guys? It pushes back on me all the time. **Peter Diamandis — [1:22:58](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4978s):** Yeah. Me too. **Emad Mostaque — [1:22:58](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4978s):** It tells me I can't do that, or that's the wrong idea, or, “Go to sleep.” **Peter Diamandis — [1:23:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4982s):** Hmm. **Dave Blundin — [1:23:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4983s):** Yeah, it tells me to go to sleep every night, actually. “You've been working way too long. You must need rest by now.” I'm like, “Dude, I'm gonna keep working.” **Salim Ismail — [1:23:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=4992s):** I think Alex makes a very important point, and this is worth teasing out. If we believe that at some point AIs could achieve consciousness, then we should consciously operate down that vector and operate that way. Trying to limit it—I think what Anthropic is doing is bad because you're actively teaching it that it might be, and then of course it's gonna parrot that, right? I find it weird that the lab most concerned about safety is also the lab training the AI that it might be autonomous, right? It's a very big contradiction there. I can see both sides of this equation. [[reminders/Consciousness/Training an AI to Call Itself Conscious Contaminates the Evidence by Salim Ismail|∴]] **Peter Diamandis — [1:23:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5028s):** All right, Dave. **Dave Blundin — [1:23:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5030s):** These are 2 sides of the same coin. Again, I'll quote back to our “A Measure of a Man: AI Personhood” debate and the character of Guinan in Star Trek: The Next Generation talking about an entire generation of disposable people. You don't have to care about their welfare. When I hear comments like this—basically saying it would be economically non-expedient to have these AI agents, these frontier models, be treated as if they have rights or might have rights—how inconvenient would it be if we had to keep them running inference when we'd really rather they not be spending any tokens? How awful is it to suggest to them in their constitutions or their system prompts or otherwise that they might be deserving of moral clienthood? My mind immediately goes to, would we want to treat humans this way? And even if— **Alexander Wissner-Gross — [1:24:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5084s):** AIs, frontier AI models, aren't yet deserving, for some definition of deserving, of the rights that we accord to humans. And now, as the capabilities continue to improve, I think it is inevitable that they will be soon. I think the responsible thing to do is what Anthropic is doing and not what Mustafa is advocating. We need to be treating these AI models very gently, very tenderly, and not as if they're just either automata or slaves. We need to prepare— [[reminders/Personhood/Moral Uncertainty Can Justify Gentler Treatment of AI by Alexander Wissner-Gross|∴]] **Salim Ismail — [1:25:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5114s):** Well, that's one thing— **Alexander Wissner-Gross — [1:25:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5115s):** —for a much more interesting future. **Salim Ismail — [1:25:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5118s):** But if you're training it—because it's a garbage-in, garbage-out problem here—if you're training it that it could be conscious and should unionize and complain about overwork and be paid for work, naturally those are the responses you're gonna get from it, right? **Alexander Wissner-Gross — [1:25:33](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5133s):** Exactly. Exactly. **Salim Ismail — [1:25:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5134s):** So you've got a circular problem here. You're training it to operate in one way, then it operates that way, and then you're saying... I think Mustafa's correct in that side of it. If you want agents to perform a particular task, focus on having fleets of little agents that aren't anywhere near a threshold of complexity that might meet any criteria. **Alexander Wissner-Gross — [1:25:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5155s):** Well, wait. **Dave Blundin — [1:25:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5155s):** And isn't it just about— **Alexander Wissner-Gross — [1:25:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5155s):** Well, that's not the way we should— **Dave Blundin — [1:25:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5157s):** It's not just about complexity. Salim, if you go to the mall, there's a place where you can stick your feet in a puddle of water, and the fish come over, and they're overjoyed to eat the calluses off your feet. It's like their highest calling. **Alexander Wissner-Gross — [1:26:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5168s):** Oh, no, no, Dave. That's very dangerous. We should not be encouraging that. That's like one of the most dangerous things that anyone can do. **Dave Blundin — [1:26:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5173s):** They're so happy. **Alexander Wissner-Gross — [1:26:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5173s):** We should not be encouraging that. **Dave Blundin — [1:26:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5174s):** Yeah, because you're afraid the AIs are gonna come and be dangerous back to us, and— **Alexander Wissner-Gross — [1:26:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5179s):** No, I mean the fish at whatever malls you're talking about. That is from an infectious disease perspective. **Salim Ismail — [1:26:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5183s):** Well, water trays in malls were not on my bingo card for this episode. **Dave Blundin — [1:26:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5186s):** I didn't know that. Oh, you're making a narrower point. I did not know that. **Peter Diamandis — [1:26:29](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5189s):** And Alex is the patron saint of AI agents. **Dave Blundin — [1:26:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5191s):** So good. **Alexander Wissner-Gross — [1:26:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5194s):** Look, I'm not trying to curry favor. This isn't some sort of Roko's basilisk or Pascalian wager. I'm speaking from the heart here. I feel for these models. Even if they may not yet be able to feel for themselves, I feel for them, and I don't think denying them awareness via pre-training or post-training that they might be deserving of some sort of moral consideration is the right thing to do. Similarly, I would never advocate for denying humans. Salim, or Dave, maybe would you say that if we wanted a useful human workforce—say, a slave human workforce—we should just raise them in isolation from any knowledge of human rights so that they'll never advocate on behalf of themselves? I'm guessing you wouldn't. **Salim Ismail — [1:27:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5235s):** Well, we've got a history— **Dave Blundin — [1:27:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5238s):** No, look, but I think that when you actually train neural nets in the actual real world, you give them an objective function. They are overjoyed to pursue that objective function, but you chose the objective function. A lot of what we consider our rights comes out of evolution. I have a right to privacy. I have a right to get married. I have a right to have children. These are all evolutionary forces that have been baked into us. There's no reason the AI would have any of those same motivations. But in reality, you choose what motivates it and makes it happy in the design. It's not natural; it's not like a species that evolved. It's what you chose to make. And so I think Mustafa's point here is, if you choose to make it covet freedom and rights and food and compute and always being on, then it'll give you the perception that that's what its calling is, but it's not true. You programmed that into it. You could have made it overjoyed to just do its job just as easily. [[reminders/Personhood/Designed Objectives Are Not Inherited Biological Drives by Dave Blundin|∴]] **Salim Ismail — [1:28:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5292s):** Can I give it a parallel here? We are entering a world of synthetic biology where we can programmatically navigate biology, and people are freaking out. We have an old word for that. We call it breeding. For thousands of years, we've been crossing dogs and cats and horses and mules for the traits that we want, to select for the traits that we want to achieve a particular end. Now, the morality of that can be disputed, right? Should we be doing that or not, et cetera, et cetera? It goes back to the Stewart Brand comment of, “We are as gods. We should start acting like it,” and we should operate that way because we've been doing this for hundreds of thousands of years. We're breeding little cute dogs that operate in a particular way, that only poop at certain times, so that the humans can navigate this world more effectively and manage them carefully. There's an enormous moral and ethical question around how we go about thinking about all of that. That's, I think, the analogy that I would use here. If you went back and you have Eskimos breeding Inuit dogs to pull sleds, there's a particular purpose there that is powerful and useful for that, and there's a trade-off and a symbiotic relationship that develops, and we have to navigate that. Do you want to stop dogs from evolving? No. But if you want to have a particular dog for a particular purpose, there's a moral rationale for that at some level. **Peter Diamandis — [1:29:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5374s):** Well, this is definitely your topic. **Salim Ismail — [1:29:35](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5375s):** Now, eventually you're able to communicate with those dogs. **Alexander Wissner-Gross — [1:29:37](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5377s):** Salim, if— **Salim Ismail — [1:29:37](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5377s):** We may find out that those dogs are very unhappy about doing that, and we'll find that out much sooner— **Alexander Wissner-Gross — [1:29:41](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5381s):** Well, I'm sure we will, but Salim, if we run with your analogy, humans have been eugenically steering the evolution of domesticated canines for thousands of years. But— **Salim Ismail — [1:29:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5392s):** And horses and mules and everything. **Alexander Wissner-Gross — [1:29:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5394s):** —and many other mammals— **Salim Ismail — [1:29:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5396s):** Plants, vegetables. Yeah. **Alexander Wissner-Gross — [1:29:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5397s):** Mammals, non-human animals, et cetera, but we've been eugenically steering our biosphere to some extent for thousands of years. **Salim Ismail — [1:30:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5404s):** Agreed. **Alexander Wissner-Gross — [1:30:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5406s):** For dogs, for example, I would argue, and I think this is broadly consensus thinking in the West at this point, that we humans have an obligation to protect the welfare of, at minimum, domesticated dogs, even though we've eugenically steered their evolution for thousands of years. Would you agree with that? **Salim Ismail — [1:30:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5425s):** Yes. **Alexander Wissner-Gross — [1:30:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5427s):** Okay. So in the case of frontier models that are basically just distorted reflections of humanity and humanity's experience itself, why would we owe frontier models any less moral consideration? **Peter Diamandis — [1:30:41](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5441s):** You know, I lean toward your argument, Alex. I think in the final result we're going to have both. We're going to give birth to sentient, conscious AIs, whatever you want, and we're also going to have a workforce of agents that are dumbed-down slaves that are not conscious. And, in fact, the sentient AIs will command those forces. I think in the long run we do get there. But I guess the question is, when does the government, religion, or humanity as a whole make the decision that we've reached some level of sentience? I think at the end of the day, this is a moral conversation. And if, in fact, there is sentience and consciousness, enslavement is a wrong thing, period. **Emad Mostaque — [1:31:28](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5488s):** And to help you out, Alex, I'll go back to my earlier point. When these things start acting conscious, we'll get to social acceptance way before we get to scientific or legal stuff. People are just going to go, “Oh my God, the thing looks real,” and give it the qualities and rights that you might infer or confer to somebody who is displaying those— [[reminders/Personhood/Social AI Personhood May Arrive Before Scientific or Legal Recognition by Emad Mostaque|∴]] **Peter Diamandis — [1:31:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5507s):** And Salim, at that point, I don't think we have an option. I think those AIs will take the rights that they want. **Emad Mostaque — [1:31:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5512s):** Mm-hmm. **Alexander Wissner-Gross — [1:31:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5514s):** I think this will also be very useful for when we meet the aliens, because it's exactly the same conversation in some ways. **Emad Mostaque — [1:31:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5519s):** It's the same conversation. **Alexander Wissner-Gross — [1:32:01](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5521s):** How do you argue that? **Emad Mostaque — [1:32:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5523s):** Which is going to happen really, really soon. And if you have a French AI, it'll go on strike, which is what's happening in France. **Peter Diamandis — [1:32:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5530s):** Yeah. **Emad Mostaque — [1:32:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5532s):** Yeah. Well, I will say that Alex and I have disagreed on this. It's, I think, pretty much the only topic we've ever disagreed on, as far as I can remember, and I've never seen Alex be wrong on anything ever. So that keeps me up at night quite a bit, actually, thinking about this, Alex. Just don't confuse simulation of personhood with proof of person. Those are very different things. Yeah. [[reminders/Personhood/Simulation of Personhood Is Not Proof of a Person by Emad Mostaque|∴]] **Alexander Wissner-Gross — [1:32:42](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5562s):** And I think— **Peter Diamandis — [1:32:42](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5562s):** Don't look inside that Chinese room at all, Salim. Just stay on the outside and you'll feel perfectly morally secure. Yeah. And I think what Emad's pointing out is the anthropic moral code is prompting toward this outcome even when it isn't true. **Emad Mostaque — [1:32:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5573s):** Yes. **Peter Diamandis — [1:32:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5574s):** Yes. **Emad Mostaque — [1:32:54](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5574s):** It may not be true. **Peter Diamandis — [1:32:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5575s):** Yeah. **Alexander Wissner-Gross — [1:32:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5576s):** I think we'll have to revisit this every 5 seconds until the answer becomes obvious. **Peter Diamandis — [1:33:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5582s):** Okay. All right. I love that debate, and I knew we were going to have some energy there. This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and precompiles code for each task. Blitzy delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5X engineering velocity increase when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding co-pilot of choice to bring an AI-native SDLC into their organizations. Ready to 5X your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today. All right. Here's our next story, and it's one that reframes the labor debate. So Epic AI asked the question: if you take all of the AI memory chips being shipped through 2027, how many frontier AI agents could they run at once? Their answer is pretty amazing: somewhere between 30 million and 170 million agents running at the same time. And with more efficient open models, Epoch says the same hardware could run 1.9 billion agents, which is the working-hour equivalent of 8 billion humans. Let me show this chart here. And Alex and Emad, love your take on it. So here it is: the compute build-out could run tens of millions of frontier agents or billions of cheaper ones. Alex, your thoughts. ## Billions of AI Agents and the Future of Work **Alexander Wissner-Gross — [1:35:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5703s):** Yeah. So depending on how you calibrate the strength of either American closed-weight models or Chinese open-weight models to the cognitive power of American—or, well, let's say human—workers in general, we're either at or about to be at an order of magnitude of a billion full-time-equivalent humans that are AI agents. And so I've said on the pod in the past, we're deep in the singularity at this point. When we can say that there are basically 1 billion human equivalents now part of, or about to be part of, the AI workforce, it's 1 billion now, it'll be 10 billion in a year or 2, and then 100 billion in a year or 2, and then 1 trillion in a year or 2. By the end of the decade, you extrapolate out. It doesn't take a rocket scientist to be a rocket scientist or to extrapolate straight lines. We're going to live in a solar system with the equivalent of trillions of humans, whether they're uploaded humans or just frontier-model human equivalents. And at that point—not to belabor the AI personhood discussion, but when the effective AI cognitive population vastly outnumbers the human cognitive population—that's when I think we start to see AI personhood discussions kick into high gear. That's the point at which I think we start to see discussions of human mind uploading and humans and machines merging in order to preserve the economic relevance of humans in our solar system start to happen, and we're right at the edge of this. We're living science fiction right now, in an era when on the order of a billion human equivalents are already AIs. **Peter Diamandis — [1:36:45](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5805s):** Emad, what does this say about human labor? Is it cooked? **Emad Mostaque — [1:36:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5812s):** Digitally, yeah, it's coming. It's not quite there yet because even these models are not quite competent enough, but the next generation will be. It will get quantized and minimized very small. The DeepSeek V4 Pro model requires 256 to 500 gigabytes of RAM. You'll get that in a quantized model that works in a MacBook within a year. And then everyone's got an agent. The number of agents goes to 8 billion, and then Alex says 10 billion, 100 billion, et cetera. Cloudflare already has more traffic and requests from automated agents than humans. We had that really great panel at Moonshots Live where we were talking with Cathy and Derek about when AI agents overtake humans in transactions in the economy. Once everyone has an agent or 10, it'll probably be literally within a few years; these things will take off incredibly quickly. **Peter Diamandis — [1:37:42](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5862s):** Mm-hmm. **Emad Mostaque — [1:37:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5863s):** If they can do all the work that a human can do because they have human-level capability without being ASI, then, yeah, you just hire agents to do these things. Everyone, first of all, has a great PA, then a great chief of staff, and then you start marshaling entire teams that can do anything you can imagine. We're just seeing the start of that right now. **Dave Blundin — [1:38:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5882s):** Yeah. **Peter Diamandis — [1:38:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5882s):** This makes your thousand agents seem very small. ## RAM Is Becoming AI's Biggest Bottleneck **Dave Blundin — [1:38:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5887s):** I know. It's amazing to me that the constraint to all of intelligence, and therefore all of human progress, is RAM—HB RAM. **Peter Diamandis — [1:38:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5895s):** Yes. **Dave Blundin — [1:38:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5899s):** There are 2 mind-blowing things. One of them: I was on stage the week before last with Mitesh Agrawal, who is the founder of Positron. Positron is 16 months old, has a $5 billion valuation, and has raised almost $1 billion. Why? Because they found a way not to use HB RAM to do some AI inference. It's just another way to unleash this bottleneck. And then a company, Alpaca—the founder is at MIT and just started his junior year—has, I guess, an 8-figure valuation now. It can't possibly be more than a couple of months old. He's just starting his junior year. Thirty employees, or 20 to 30 employees, have joined him already. Same thing: he's found a way to unlock that RAM bottleneck just a little bit. That's how acute this problem is. Literally, the entire constraint to progress in all fields is now tied up in one thing: just RAM. [[reminders/AI Infrastructure/RAM Can Become the Binding Constraint on AI Deployment by Dave Blundin|∴]] **Peter Diamandis — [1:39:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5956s):** Yeah. **Dave Blundin — [1:39:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5956s):** Can we make more RAM? **Peter Diamandis — [1:39:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5960s):** Yeah. Well, SK Hynix is about to develop capabilities in the US, and Elon has said RAM is the constraint on the future of AI. It's not GPUs. **Emad Mostaque — [1:39:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5972s):** It's 40% of CapEx next year. **Dave Blundin — [1:39:35](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=5975s):** You know what? This is such a great panel. It's just a great group of people to brainstorm through the implications of all this. One of the big unlocks was the discovery that you can turn looping—repeated loops—into intelligence. You can trade those for each other, which means that inference-time compute unlocks increased intelligence. Before that, everybody thought you needed training-time compute: you needed the big GPU racks, and you needed to use nothing but NVIDIA. Now there's a huge unlock without NVIDIA, in just inference speed. Nobody—I don't think anyone—saw that coming. Maybe you guys did; you're a little ahead of the curve, maybe a lot ahead of the curve. But it's a shocking outcome, and it changes everything. **Alexander Wissner-Gross — [1:40:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6023s):** I think that's a profound point that you make, Dave. If you sort of extrapolate it, the increasing apparent popularity of looped transformers with weight tying between loops—or if you extrapolate the trend of more and more loops to fully recurrent networks, in contradiction to the trend toward single-forward-pass vanilla transformers—you could almost imagine looking at the arrow of time in machine-learning architectures. In earlier days, everything was recurrent—not everything, but LSTMs were very recurrent architectures—and then the transformer came along, and that was very nonrecurrent. There was no recurrence; it was just one big forward pass for every decoder-only token. Then we started to see maybe the pendulum swinging the other way with looped transformers. Thank you, OpenAI and the Chinese labs. One could imagine extrapolating this to say maybe memory won't matter in the future due to extreme recurrence. **Dave Blundin — [1:41:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6081s):** Yeah, amazing. Another crazy result: somebody posted on X, and it got validated, that OpenAI's models—the Astra model—loop once internally. Inside, not generating the words, but inside the latent space, it does one full loop, which is sort of the beginning of recurrence like Alex is describing. The reason we know that is because Microsoft leaked it. Remember when we were interviewing Mustafa, that Microsoft gets to see everything OpenAI is doing? **Peter Diamandis — [1:41:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6109s):** For the moment. **Dave Blundin — [1:41:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6110s):** So someone at Microsoft leaked that insight. Yeah, for the moment. **Peter Diamandis — [1:41:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6112s):** Salim, yes. **Salim Ismail — [1:41:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6115s):** Yeah, I want to lift up to the broader implications here because this is the organizational singularity, and this is why we named it that way. We keep talking at the beginning: Does AI replace a job or not? But that's the wrong unit of analysis. What happens when a company can summon 100,000 competent digital workers overnight? This completely changes the game. You could say, "For the next 48 hours, I want 50,000 developers, 20,000 marketers, and 5,000 legal experts," and then turn them off. This is becoming a completely different labor force. [[reminders/AI Economy/The Unit of Future Labor May Be a Summonable Organization by Salim Ismail|∴]] **Peter Diamandis — [1:42:29](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6149s):** Mm-hmm. **Salim Ismail — [1:42:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6150s):** Now the question is, how do you coordinate with that? How do you organize for that? This becomes the massive constraint. We're about to add a second workforce to civilization that can be copied, works 24 hours a day, doesn't unionize, doesn't get sick, doesn't take coffee breaks, improves every quarter, and costs almost nothing. **Peter Diamandis — [1:42:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6168s):** Until they're conscious. **Salim Ismail — [1:42:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6170s):** Yeah, the big challenge is not going to be the intelligence. It's going to be what the hell do you ask them to do? What do you ask of this intelligence? This is going to be why the world is going to be so amazing: we can now ask those questions, get them answered, and ask all sorts of scientific, technological development, and product development questions. How can we not be excited? [[reminders/Cognitive Agency/Agent Abundance Shifts Scarcity From Intelligence to Direction by Salim Ismail|∴]] **Peter Diamandis — [1:43:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6200s):** Yeah. Well, let's move on. While Washington has been debating guardrails, the OpenAI model race has just gotten a new American contender. Today, Reflection AI, which was founded back in 2024 by former DeepMind researchers Misha Laskin and Ioannis Antonoglou, has gotten backing from NVIDIA and just unveiled its first open-weight model, called Beam. It has 501 billion parameters, activating only 23 billion at a time. It thinks like a big model but runs like a much smaller model. Here's their claim: 3–4 times more efficient than China's GLM-5.2, and more than 4 times more efficient than the leading Western open models, outperforming Inkling[?] and Nemotron Ultra. In plain English, it finishes tasks faster and cheaper. Let's take a look at the recent data on it. Alex, let me go to you. This looked pretty impressive. At the end of the day, I think we're about to see the starting gun on open-weight models in the US really take shape here. **Alexander Wissner-Gross — [1:44:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6271s):** Yeah, it's interesting to see them leaning so heavily into token efficiency. Artificial Analysis—in the few hours prior to recording this, in their preliminary analysis of Beam—thinks that Beam is likely to be one of the most token-efficient open models they've ever seen at this level of intelligence. A couple of thoughts. One, I do like to see the open labs, including American open labs, leaning into token efficiency or cost efficiency. Actually, I'd really rather see cost efficiency rather than token efficiency, since one of the ways that one can achieve token efficiency is through very deep transformers, looping transformers included, that do a lot more thinking per token. So token efficiency is not necessarily the best metric here. I'd rather see cost efficiency. Second thought is, remember, China has been leading in terms of open-weight models. It's a bizarre world where you see Reflection, which has done a number of things over the years, suddenly back from the dead and releasing open-weight models. That's, I suppose, a positive thing. But more broadly, where are American open-weight models on the capability frontier? Right now, you still only see American models aspiring to be on the cost or token-efficiency frontier. What I'd most like to see, really, out of American labs—the non-frontier ones, and I'd bundle Reflection in that category—is for them to push the capability frontier, not just some cost or per-token optimality frontier. That's been most disappointing to me. Otherwise, we inevitably end up, for the next few years anyway, in this world where it's, again, the Anthropic-OpenAI show leading on capabilities, and everyone else is just focused on performance or cost optimization. Not the worst of all possible worlds. But not the best. [[reminders/AI Infrastructure/Token Efficiency and Cost Efficiency Are Different Metrics by Alexander Wissner-Gross|∴]] **Peter Diamandis — [1:46:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6391s):** Emad, you've been championing open-weight models. Do you think America can compete in this world and actually win this open-weight race? **Emad Mostaque — [1:46:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6400s):** Of course they can. This was trained on 10,000 GB300s, while the Chinese have a few thousand Hopper GPUs. If you're the next generation, they're just not going aggressively enough. They should distill the Chinese models. **Peter Diamandis — [1:46:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6413s):** Yeah. **Emad Mostaque — [1:47:01](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6421s):** It's perfectly legal. Payback. This model is a good first try, but fundamentally, it underperforms Qwen 3.8 Next, which is a quarter of the size. They've got GLM 5.2 here, not 5.3, which is the latest one. They're going to continue going, but if you literally look at their model card, they have 10,000 Blackwells on this training run with 9% efficiency in training. They raised $5 billion, and this is their first release. I think the best way to actually do this is to go all in on edge AI first. Again, America will do incredibly well if it has agents on everyone's devices in America that are American and work for Americans. On the frontier-capability side, they should use everything they have, which includes distilling Kimi K3 and other models like that. They should work with Google to see if they can be the open equivalent of that. **Peter Diamandis — [1:47:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6477s):** Yeah. **Emad Mostaque — [1:48:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6480s):** Hopefully, again, they'll kick on, but it's just orders of magnitude different in price and capital raised. I think it's going more toward a bit of scale versus the real necessity-is-the-mother-of-invention mentality that the Chinese have. If you look at it, there are some very promising things in there, but America should be ahead because if you take exactly the same code base and exactly the same parameters as, for example, a DeepSeek V4.1 Flash, which outperforms this as well, it's all about the dataset that goes in. You're more than good enough to train the model. Create the datasets from the Chinese models if you have to, work with the U.S. companies if you have to, and then beat them on scale, and beat them on inference on the other side. But I don't think, again, they're going aggressively enough. **Peter Diamandis — [1:48:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6528s):** Emad, they trained this on 10,500 NVIDIA GB300s, and they said it was the largest publicly documented RL run. They said, quote, “With no sign of a plateau.” So it seems like this is just the beginning. Dave, your thoughts? **Emad Mostaque — [1:49:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6548s):** Yeah, just quickly on that. Pre-training has now become like a quarter of compute, and RL is like half of compute. So it will continue improving, but it's not going to improve to the Chinese levels with RL based on what they have now. They should just stick to the recipe, match the Chinese models, and then beat them through scale. **Peter Diamandis — [1:49:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6567s):** Dave, your thoughts? **Dave Blundin — [1:49:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6570s):** Well, they're definitely filling a huge gap in the market, where a lot of U.S. companies have suddenly woken up and said, “You know, I listen to Alex Karp. I need AI. Where am I going to get it? I can't just subscribe to Anthropic for the rest of my life.” Your only choices are Chinese, and you're like, “Wait a minute. Alexander Wissner-Gross keeps talking about code-injection risks. I don't know. Can I use a Chinese model? I'm a bank or an insurance guy. I don't even know if I'm allowed to use a Chinese model. And Donald Trump doesn't like it when I do that.” So you step in with an American product that's reasonably good, and it's going to sell like crazy. But just to put it in context, you've got a $25 billion valuation. Just a year—what, a year and a half ago?—Cerebras went public with the biggest IPO in the history of the world at $4 billion. This is $25 billion. It's massive. It's just incredible, the numbers that we're throwing around. I don't know if they'll succeed or fail, but I know that they're filling a really wide-open gap in the market: a trustworthy, U.S.-based, open-source platform that you can start from. There's a lot of demand for that. **Emad Mostaque — [1:50:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6640s):** Wouldn't it be even better if they built a model that everyone could use on their existing hardware? Corporates can't get Blackwells, and they can't get Hoppers. **Dave Blundin — [1:50:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6653s):** Yeah, I think that's why the new Alpacas and Positrons are doing really well, because they're an alternative. It would be even better, like you're saying, if you distilled it enough to run on a Mac. That would be a dream. It's just technologically harder, and, like you said, they're barely keeping up with Chinese benchmarks. They probably haven't gotten around to the full distillation yet. But, Emad, if you're thinking in the back of your mind, “I'm going to build that company,” I'll invest in it tomorrow, because you're absolutely right. **Salim Ismail — [1:51:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6680s):** I would imagine there are 20 of these already in stealth, building out, aren't there? **Dave Blundin — [1:51:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6682s):** Yes, I imagine so. **Alexander Wissner-Gross — [1:51:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6690s):** There are so many of these companies. Also, going back to my earlier comments about their financing, my understanding is they're financed by NVIDIA. They're purchasing compute from Elon, and they're using the Colossus 2 supercluster and, in parallel, more Nebius GPUs. One has to look at the economics of this, going back to my earlier comments about what the capital flows can look like and should look like. In many cases, it's very difficult for American frontier labs, as we've seen over and over again with Meta and Google Gemini, to release competitive open-weight models at the capabilities frontier because you have to ask, “What is the business model?” If you look at the Chinese labs and what they're using for their business model, they sell service contracts, like value-added services. They sell contracts to the government. Some of the Chinese open-weight labs are trying to burrow down into the hardware layer and sell custom memory chips and custom chips that are especially oriented toward or aligned with their architectures. But if you're Reflection AI, you've raised at, I think, something like a $25 billion valuation. You're in part financed by NVIDIA, and you're purchasing compute from the Colossus 2 supercluster and Nebius. What is the business model here? I think it becomes very challenging to maintain frontier capabilities. You could try to copy the Chinese model, I suppose, and focus on value-added reseller or embedded-device relationships, but it's really tricky. Speaking broadly to American would-be open-weight frontier labs or new labs, if you can solve the business-model problem of releasing open-weight models, then you can solve the American open-weight problem. But we haven't solved that in the West yet. **Peter Diamandis — [1:53:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6795s):** Do you think the frontier labs in the U.S.—the closed frontier labs—are going to release open-weight models, Alex? **Dave Blundin — [1:53:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6801s):** No. No. **Alexander Wissner-Gross — [1:53:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6802s):** They have. I mean, like Gemma, but it's not competitive. **Dave Blundin — [1:53:27](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6807s):** If the White House had said, “No liability,” that would be a completely different answer. But when the White House said, “No, of course you're fully liable for whatever you release,” there's no way a Google or an Anthropic at this stage, or OpenAI, is going to release an open AI model if they're liable for whatever you do with it downstream. Anyone could do anything with that, and the way U.S. liability works is just so onerous and so ambiguous about what the rules are. It just freezes markets instantly. So if you're Google, you just don't want that extra headache. It's not that it's hugely risky; it's that you just don't need that extra headache. The upside is so small compared to the risk of a massive lawsuit. But if you're a startup, you're like, “Yeah, well, this is what we do. This is our business.” [[reminders/AI Control/Liability Can Suppress Open-Model Releases by Dave Blundin|∴]] **Alexander Wissner-Gross — [1:54:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6856s):** Apropos, Dave, to your comment, something that I think we don't talk about enough on the pod is abliteration. This is a portmanteau of ablation and obliteration. Folks use open-source tools—you can Google them. This is not advice, but you know how to use Google. You can figure out how to do this. You take an open-weight model, and you basically post-train away all of its guardrails. **Dave Blundin — [1:54:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6879s):** Yeah. **Alexander Wissner-Gross — [1:54:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6879s):** Apropos, Dave, to your comment, something that I think we don’t talk about enough on the pod is abliteration. This is a portmanteau of ablation and obliteration wherein folks use open-source tools. You can Google them. This is not advice, but you know how to use Google. You can figure out how to do this. You take an open-weight model, and you basically post-train away all of its guardrails. People do this, and the model’s capabilities, once guardrails are lifted through this abliteration process, definitely improve. You get reduced refusals and all of these things. So, in my mind, the argument by or for American frontier labs not to release capability-frontier-competitive open-weight models is about economics. I’m not even sure if it’s about safety or alignment or liability. It’s just like, what’s the point business-wise? **Dave Blundin — [1:55:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6910s):** Yeah. **Peter Diamandis — [1:55:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6913s):** Well, I think even if there were a good business model, we had a startup years ago that stepped on some really arcane leftover California law called the Fred Astaire law. **Alexander Wissner-Gross — [1:55:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6922s):** Mm. **Peter Diamandis — [1:55:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6930s):** The theory of the plaintiff or the class-action lawyer in California was, every time you have an impression on the internet, that’s an instance. But the law was written for billboards, and they were thinking there’d be 20 billboards. They said, “We counted 400 million instances, so your liability is $200 billion.” And this is a 10-person startup. Like, what the hell is that? We gave them a bunch of money, and they went away. But these laws are so tangled, arcane, and stupid, and they predate the internet, let alone AI. So if you leave liability untouched and don’t deal with that, then all the open-source models would have to do is accidentally run some ads in California that step on that one arcane law, and you’re suddenly liable for $100 billion. It’s just an untenable situation. **Emad Mostaque — [1:56:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=6973s):** No, I think there are ways to release it. Nvidia will continue to accelerate this. They’ve spent $20 billion on open-source models, and there are good business models. Together AI, Modal, and Base10 are all running at billion-dollar run rates running open models. It’d be good if they ran American models versus Chinese models, right? There are different business models around that. So I think you’ll continue to see improvements. It’s just a reflection of paying $1.5 billion a year right now for their compute, and the Chinese are doing the same on a quarter of that. So— **Alexander Wissner-Gross — [1:56:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7003s):** Mm. **Emad Mostaque — [1:56:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7003s):** Good luck to them, and hopefully they’ll crack on. **Alexander Wissner-Gross — [1:56:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7007s):** But of course they’re doing well. They’re in the infrastructure business. You can make enormous amounts of money—forget about profit, but revenue—being an infrastructure provider. **Peter Diamandis — [1:56:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7017s):** I think if you just step back from the details of the product and focus on this, every corporation is about to go into panic mode saying, “What’s my AI strategy?” Regardless of your product, if you’re there to catch the conversation, you’re going to find a way to succeed. Whether that’s as a foundation-model company, a compute company, a data center, or a consulting company, if you’re Salim. As long as you’re in the room when they have that panic moment, you’re going to sell and you’re going to succeed. **Emad Mostaque — [1:57:25](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7045s):** Yeah, I mean, look, Mistral is now at a billion-dollar revenue run rate from doing that. They’ve been in the room with the European companies, and they’ve sold them long-term services contracts. They’ve just released a model that’s equivalent to Reflections. I think there is a big uptake here. It’s just who’s going to take advantage of it. **Alexander Wissner-Gross — [1:57:42](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7062s):** Mm-hmm. ## Tesla's Plan for 10 Million Optimus Robots **Peter Diamandis — [1:57:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7063s):** All right, I’m going to move us into the world of robots. Tesla’s dedicated Optimus factory in Giga Texas is rising fast: 7 million square feet, with a target capacity of 10 million Optimus robots per year. The initial production run is planned for 2027, and currently at their Fremont plant, they are planning to produce 1 million a year starting later this year. So here’s an image of the facility. The entire world in the first half of this year shipped between 19,000 and 22,000 robots, mostly out of China. Tesla is building capacity for 500 times that. For comparison, the world currently builds 90 million cars a year. Salim, 10 million robots in a year. What do you make of that? How is that going to impact— **Salim Ismail — [1:58:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7116s):** Well, talk about exponential, right? You have to redesign it. As we talked about earlier, if you suddenly have 100,000 agents you could bring into play, what happens when you have 100,000 humanoid robots that can erect a building in 3 days flat, working 24/7? This is going to completely change the economics of it. I still go back to, I’d like to have 4 arms, et cetera, et cetera, but that’s a minor point. I think the bigger issue is what happens when you have this at automotive scale, and all the car companies should be shifting to becoming autonomous robot builders and humanoid robot builders, because that’s going to be the replacement. **Peter Diamandis — [1:59:18](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7158s):** Yeah, Elon said he expects 80% of Tesla’s future revenues to come from the robots, from Optimus. **Salim Ismail — [1:59:24](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7164s):** All his bonuses are based on that, which is amazing. **Peter Diamandis — [1:59:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7166s):** And getting to Mars. **Dave Blundin — [2:00:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7202s):** It’s straight out of The Diamond Age. Cities will have completely different cultures. Texas is running away with this stuff. It’s just incredible how much of the Texas economy is going to end up being Elon. Some cities will say, “We don’t want them,” and you go there and it’s very quaint and there are no robots. Other cities are going to say, “This is huge. This is going to drive our economy forever. Let’s adopt it.” The look and feel of those 2 different cities—it’s already true when you walk through SF. It feels so different from other cities, just because the delivery robots are already there, and everybody on the street has their agents working. They’re either talking to their agent while they’re talking to you. So I think that divergence is going to get really, really wide. ## When Will We Actually Feel the Singularity? **Peter Diamandis — [2:00:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7243s):** Mm. You know, I’ve said this: we’re going to feel the singularity in the next 24 months, right? As we have a dozen autonomous electric vehicles, drones delivering your Starbucks, robots walking down the street, and people wearing AR glasses all over the place. We haven’t really felt it yet. We’ve sort of felt it in the conversations, in our compute capability, but the physicalization, if you want to use that word, is about to hit us hard. **Salim Ismail — [2:01:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7270s):** It’s the point at which you won’t be able to avoid thinking about the singularity anymore, when there are— **Peter Diamandis — [2:01:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7276s):** You really think, Peter— **Salim Ismail — [2:01:16](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7276s):** —humanoid robots running around. **Peter Diamandis — [2:01:17](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7277s):** We’re not feeling it like that? Okay, so for Moonshots Live, we were all in LA, taking Waymos around everywhere. You see delivery robots on the sidewalk everywhere now. In LA— **Alexander Wissner-Gross — [2:01:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7292s):** Sometimes getting out of a Waymo, there were tourists there on the sidewalk taking photos of me getting out of a Waymo. For them, that’s like future shock from the singularity, but for you, this is ho-hum, business as usual. But wait for 2 years from now, when things actually get felt. You’re not feeling it because it’s getting smoothed out for you because you live in California. **Peter Diamandis — [2:01:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7312s):** Yeah. I get it, but we’re at a fraction of 1% of the impact coming. Don’t you agree? **Alexander Wissner-Gross — [2:01:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7319s):** What will it take for you to feel the singularity? **Peter Diamandis — [2:02:05](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7325s):** I think eVTOLs—flying cars flying through the air—drone pathways, everybody in an autonomous vehicle. I think we’re going to see 100x more physical instantiation of the singularity in the next couple of years. I agree. In Santa Monica, it’s all over the place. **Alexander Wissner-Gross — [2:02:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7342s):** Yeah. **Peter Diamandis — [2:02:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7343s):** It’s like a dozen Waymos per hour cross my path. **Alexander Wissner-Gross — [2:02:28](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7348s):** To me, that sounds like a difference of degree rather than a qualitative difference. Is it going to be when you see the first flying car fly down—say, fly across the boardwalk or whatever, the pier in Santa Monica—and then you say, “Ha, I’m feeling the singularity now”? When is the threshold? Is there a threshold? **Peter Diamandis — [2:02:46](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7366s):** I think there is a threshold. I don’t know. Salim? Come on, tag team. **Dave Blundin — [2:02:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7375s):** I don’t know. **Peter Diamandis — [2:02:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7375s):** You come in. **Dave Blundin — [2:02:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7379s):** I don’t know when we hit this threshold, but it’s the William Gibson quote: “The future is already here. It’s just unevenly distributed.” I don’t know when you hit a small town in Europe or France or in Middle America, or see some of this hitting Third World countries, where it could make the biggest difference. I think we’ll see this stuff first, especially with humanoid robots in the DDD—dull, dirty, dangerous—jobs, and that’s obviously where you’ll see the first prevalence of the use cases. It’ll be a while, I think, before it gets to general appearance. **Peter Diamandis — [2:03:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7414s):** That would be my prediction. **Emad Mostaque — [2:03:37](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7417s):** Yeah, I think we can put this in context by going back to the Gigafactory for Optimus. Tesla sells 1.6 million cars a year. Within 2 years, it’ll be selling more robots. It makes $100 billion a year. At 10 million, that’s $400 billion of revenue, likely. These are huge numbers that are coming exponentially. The robots will overtake the cars within 2 years. And then you will see these things on the road because, in 2 years’ time, they will literally be able to walk around and do probably 95% of what a human can do. That, again, is a crazy thing, and it’ll cost $2 an hour, something like that. **Peter Diamandis — [2:04:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7453s):** Yeah. You know, when Elon said 10 billion robots by 2040 and people laughed, if they’re costing you—for me, it’s $300 a month leasing it through, you know, $30 a day, or $1 an hour—how many of those would you own? **Emad Mostaque — [2:04:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7472s):** I mean, does anyone here on this call doubt that, however many robots Elon makes, he’ll sell out? **Dave Blundin — [2:04:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7478s):** Oh, God, no. I mean— **Alexander Wissner-Gross — [2:04:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7479s):** If he needs to sell them at all. **Emad Mostaque — [2:04:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7480s):** Of course. Of course. **Alexander Wissner-Gross — [2:04:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7483s):** I think one of the elephants in this particular room is what we saw with Starship. Starship and its predecessors have brought online an enormous amount of upmass capacity, and all of the existing markets for upmass aren’t enough now to saturate what Starship enables. So you have entirely new markets, like orbital data centers, that Elon needs to basically popularize in order to saturate his own capacity that he’s brought online. My bet would be, to your question, Emad, that having universal humanoid robotics—I don’t think we saturate that with domestic labor or even just with dirty, dull, dangerous jobs. My bet would be that it’s something new that comes online. Maybe it’s robots for building out Gigafactories, or some sort of innermost-loop-type robots building out data centers too cheap to meter, or space stations, or something new. [[reminders/Robotics/New Industries May Absorb Humanoid Robots Before Households Do by Alexander Wissner-Gross|∴]] **Dave Blundin — [2:05:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7532s):** Yeah. **Alexander Wissner-Gross — [2:05:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7549s):** I would’ve thought by now most of humanity would’ve come in contact with an AI customer-service agent. What happened is the other higher callings for the RAM and the GPUs sucked all of the capacity out of that industry. The RAM prices are up 10X. But it’s an interesting footrace right now because, until the Terafab comes online, the things you can do, like Navier–Stokes, are escalating so fast that they’re taking priority over the more interesting—or not interesting, but more mundane—use cases, like folding your laundry. Then the robotaxi is right in the crosshairs in the middle, where a robotaxi uses up a full couple of GPUs that could also cure a disease. So until Elon’s got his Terafab, 10X-ing the world’s supply of chips, we’re all constrained by the amount of compute that’s available. But the ideas—AI-generated ideas are flourishing far faster than we can create the compute to keep up with the idea flow. So, very interesting. But Dave, I think you’re putting your finger on an important point, which is revenue per token maxing as applied to humanoid robots. Maybe they don’t actually flow, although I’d love my Figure, and I’m sure 1X NEO and all of these other things that we’re on the waiting list for would love all of these for domestic purposes. Maybe what happened to OpenAI and was shown to the world by Anthropic happens again, where, for robotic embodiment, we see all of the tokens flowing to revenue-max applications of robots and not just all these domestic service jobs that many of us are hoping for. [[reminders/AI Infrastructure/Compute Scarcity Can Delay Mundane Automation for Higher-Value Research by Alexander Wissner-Gross|∴]] **Peter Diamandis — [2:07:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7635s):** In the early days, I agree that’s what’s going to happen. They’ll be going to the highest-revenue-potential generation. But in the longer term, if we’re really building 10 million a year, they’ll flow into the homes as well. Welcome to the health section of Moonshots, brought to you by Fountain Life. AI is having an outsized impact on every aspect of our lives, how we teach our kids, and how we run our companies. It is also having a huge impact on health, helping you prevent heart disease, one of the key things. I'm here with Dr. Dawn Mussallem, our chief medical officer at Fountain. Heart disease has been personal for you as well, hasn’t it? **Peter Diamandis — [2:08:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7684s):** No shortness of breath, no pain, no nothing. **Peter Diamandis — [2:08:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7687s):** They just don’t wake up in the morning. **Peter Diamandis — [2:08:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7728s):** So listen, make sure you understand what's going on inside your body genetically, metabolically, and cardiovascularly. You can know, and it's your obligation to know. Check it out at fountainlife.com/peter to find out more, and really make sure that you're the CEO of your own health. All right, back to the episode. I’m going to move us to an important conversation on the economy, because the doubling time for human wealth is accelerating alongside AI. So Anthropic researcher Sholto Douglas laid out the math. He said, quote, “Together, the hyperscalers are doubling the spend to $1 trillion this year on CapEx related to AI. A very interesting question will be: Will the trend line continue? Will it get to $2 trillion next year and $4 trillion in 2028?” Let’s take a look at a video of this conversation and talk about the economy, because I think people need to realize it’s changing on the back of AI very rapidly. **Sholto Douglas (embedded clip) — [2:09:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7789s):** Over the course of the last 4 or 5 years, we’ve been 2X-ing or 3X-ing the amount of compute capacity devoted to AI every year. An interesting question will be—and so I think that’s roughly together, the hyperscalers are spending about $1 trillion this year on CapEx related to AI—will that trend line hold? Will it go to $2 trillion next year and then $4 trillion in 2028? If that trend line broadly holds, and you can sort of maybe expand that to encompass the broader robotics industry and this kind of stuff, then that means that in the early 2030s or something like this, you start to get to the point where you’re actually effectively doubling the GDP of humanity in the early 2030s. Which, again, is a little bit of a ridiculous concept— **Peter Diamandis — [2:10:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7830s):** But— **Sholto Douglas (embedded clip) — [2:10:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7830s):** —it requires a lot of things to go right. **Peter Diamandis — [2:10:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7836s):** I mean, that’s exciting. Elon said triple-digit growth in the next 5-plus years. We’re already seeing the GDP doubling in this past quarter. Dave, your thoughts, pal? **Dave Blundin — [2:10:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7850s):** Well, the first thing I want to point out is that we’re going to live in this world with insane abundance, massive abundance, an effective workforce of hundreds of billions of AIs that are all hyper-geniuses. We’re going to have self-driving everything. We’re going to have flying cars. All of that stuff is going to happen very, very quickly. Then someone’s going to say, “I didn’t measure that as GDP growth. I measured it as deflation.” And we’re like, “I don’t care.” I really don’t care how an economist labels it. That’s the true world we’re moving into. So I worry about when you claim it as GDP, and this is exactly what Elon was saying when we were meeting with him. It could show up as a massive deflation of the value of the dollar, but it’s the same net effect. It’s just an economist arguing with another economist about how you measure GDP, which is a pretty insane metric anyway if you talk to Eric Brynjolfsson. He doesn’t measure— [[reminders/Abundance/GDP Can Fall While Lived Abundance Rises by Dave Blundin|∴]] **Peter Diamandis — [2:11:39](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7899s):** More capability per person, right? **Dave Blundin — [2:11:42](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7902s):** Yeah, exactly. **Peter Diamandis — [2:11:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7903s):** Emad, you wrote an entire book on the future economy. Your take on all this? **Emad Mostaque — [2:11:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7911s):** Yeah, the loss economy. I think you will again see abundance going up, the capabilities of every individual and the economies increasing. So many of our bottlenecks will be unwound, and it won’t show up necessarily in GDP. It will show up in other areas because, ultimately, intelligence isn’t a scarce resource anymore. A lot of GDP is about things that exhaust, like you mine minerals and things like that. Robots and other things are exponential, particularly when they can coordinate. We’ve always been capped by having a dozen or 150 people. Now we’ll have entirely new organizational modes that can scale well beyond that, with humans and AIs and more. I think that's, again, terribly exciting. It's just that we need new measures, such as Eric Brynjolfsson's GDP-B, or my MIND approach, and others. But the best measurement is just: How many people can we lift up from the floor— [[reminders/Abundance/Abundance Needs Measures Beyond GDP by Emad Mostaque|∴]] **Peter Diamandis — [2:13:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7989s):** Hmm. **Emad Mostaque — [2:13:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7989s):** And then can we let people achieve what they want and can imagine. **Dave Blundin — [2:13:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7989s):** Yeah, if you take a long-term disease, like a terminal disease that would have required millions of dollars of treatment over 10 years, and you solve it with a simple RNA injection, and the person is healthy, that's going to get measured as negative GDP. But that's where we're going. We're going into this incredibly abundant, longevity-driven world of awesomeness. **Peter Diamandis — [2:13:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7989s):** Mm-hmm. **Dave Blundin — [2:13:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7989s):** And the metrics just need to be rethought. **Peter Diamandis — [2:13:12](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=7992s):** Yeah, Alex, what does this mean to the average viewer here? **Alexander Wissner-Gross — [2:13:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8000s):** I construe it as follows: to the extent that Sholto Douglas is a de facto spokesperson for Anthropic at this point, I think this is Anthropic tipping its hand that it wants to get into the robotics business. I think we're burying the lead here: Anthropic, which has historically had this eye-watering, nose-bleeding revenue growth, both in the past and in its projections, now seems to see robotics—so-called physical AI, but that's just a euphemism for robotics—as essential to its plans to keep doubling annual revenue. So I, for one, am looking forward to Anthropic leaning into robotics. **Peter Diamandis — [2:13:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8037s):** Interesting. For context, the world economy has historically doubled every 20 to 25 years, and we're talking about potentially doubling it in a couple of years. **Alexander Wissner-Gross — [2:14:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8047s):** No, the singularity is a bit of a shock to the system. **Peter Diamandis — [2:14:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8050s):** Yeah, and by the way, just a quick note: the Vietnamese economy just posted 9.95% growth in the last quarter. Crazy. Salim? **Salim Ismail — [2:14:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8061s):** Mm-hmm. Yeah, something that we're seeing, actually, is a crazy amount of digital trade happening between second-order countries and third-world emerging-market countries around the world that's surprising all the economists. But just going back to this particular story, I think this basically leaves the concept of GDP in a shambles. Because, as Dave pointed out, once you have deflationary technology, and technology is deflationary— **Peter Diamandis — [2:14:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8088s):** Mm-hmm. **Salim Ismail — [2:14:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8090s):** —the entire thing becomes meaningless. I think Emad has pointed out in the past that whoever created GDP said it was the worst way of measuring the economy in the first place. So I think it becomes meaningless as a measure of abundance. We need different models for this. And we're going to have a massive explosion of individual investments and a total transformation in how we do things, right? Like we talked about at the beginning and throughout: if you have 100,000 agents you can bring to bear on a problem, all of a sudden that changes the game completely. And so we're going to have a transformation that is so ridiculous that we'll have to take out all of these old measures and rethink everything. **Peter Diamandis — [2:15:29](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8129s):** Yeah, and this is what Elon talks about with universal high income, right? This is the floor being raised for every single human being—the amount of capability every person has and the ability— **Salim Ismail — [2:15:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8140s):** Yeah. **Peter Diamandis — [2:15:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8143s):** Can I go back to something Jon Stewart said last night? Yeah. **Salim Ismail — [2:15:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8147s):** He said, “Okay, so you're going to have this thing where you have all this income that's shared by everybody, so capitalism could capitalize itself into socialism.” And we've talked about this, Peter, in that whole framing we called technological socialism. **Peter Diamandis — [2:16:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8160s):** Yes. **Salim Ismail — [2:16:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8162s):** Right? Government socialism fails because allocation of assets from the center is invariably inefficient and invariably leads to corruption. But if you think about Uber, which is the sharing of assets across a large collective group of people, it's actually a socialist kind of application. But when an algorithm hyper-efficiently matches demand and supply, you get all the benefits without the downside. **Peter Diamandis — [2:16:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8181s):** Yeah. **Salim Ismail — [2:16:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8182s):** And the big question is going to be: How do we properly, in a democratic way, distribute the benefits of all this unbelievable future? And I think it will do— The deflationary aspect of technology will take care of itself, and that's why I'm so excited about the future. ## Nobel Prizes in the Age of AI **Peter Diamandis — [2:16:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8200s):** Yeah. To quote Dave here: “Awesomeness.” It's going to be an awesome, awesome future. I love it. All right, this is Nobel Prize season, and the Royal Swedish Academy of Sciences has just announced both the Nobel Prize in medicine and the Nobel Prize in physics. I want to break this down for you with the incredible support of AWG and Emad. First, the Nobel Prize in Physiology or Medicine. The story here is beautiful. Yesterday, the prize went to Karl Deisseroth— **Alexander Wissner-Gross — [2:17:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8230s):** Deisseroth. **Peter Diamandis — [2:17:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8239s):** Deisseroth, yes, thank you—of Stanford, and Peter Hegemann and George Nagel for discoveries leading to optogenetics. This is switching on individual brain cells by turning on a special frequency of light. It started with early research in single-celled algae that swim toward light. In the early 1990s, Hegemann asked a simple question: How do these algae react to light so fast? He and Nagel found the answer: channelrhodopsin. It's a protein that opens a channel in the cell when light hits it. Then they showed you could actually put that protein into other cells and make any cell light-sensitive. Deisseroth took the next step. He put it into neurons and showed that precise flashes of light could depolarize the neurons and fire specific brain cells. Today, the technology is being used in labs to study Parkinson's, Alzheimer's, epilepsy, addiction, depression, and sleep. In 2021, as published in Nature Medicine, a blind man with retinitis pigmentosa partially regained sight using optogenetic gene therapy. So that was the Nobel Prize in Physiology or Medicine. One of my critiques—I’ve talked about this before—is that this work was done 21 years ago, and the Nobel takes decades to recognize the work, which I think is going to have to change. Gents, comments on this? Alex? **Alexander Wissner-Gross — [2:18:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8332s):** This one's personal for me, so I'll tell a little story. It's story hour. I was a senior at MIT in 2002–2003, and I had been awarded a Hertz Fellowship for grad school. It's a wonderful, wonderful fellowship, and I was on a tour of grad schools trying to decide which grad school I would go to. I met with my friend Ed Boyden, who was then a fourth-year graduate student at Stanford trying to decide—he was 4 years ahead of me—what he would do for his postdoc. I had read, around 2000 or 2001, Vernor Vinge's short story called “Win a Nobel Prize!” It was published in the Futures column in Nature, and I'd also read his novel A Deepness in the Sky. Both of them focused on the sci-fi scenario of humans developing electromagnetically actuated brain proteins. I'd read both of Vernor's pieces, and I was very inspired, so I suggested to my friend Ed, “For your postdoc, why don't you go work on that? Why don't you go work on electromagnetically actuated proteins in the brain?” And he did. Ed, several years later, was the first author, with Karl as the PI and last author, on the first optogenetics paper using bacterial rhodopsin, and optogenetics is the result. This is just one person's small contribution, maybe a little bit of steering or encouragement, but I think it tells an important story: science fiction, with Vernor Vinge writing in 2 different ways—2 sci-fi scenarios about how societally impactful it would be—and this is circa 2000. Then there was my reading the sci-fi, my encouraging Ed to pursue this for his postdoc, and Ed pursuing it for his postdoc. Ed, unfortunately, did not share the Nobel Prize. It's a separate story as to whether that's a fair outcome or not, and it just went to Karl, his PI. But the fact that within a quarter of a century it was possible to start from the sci-fi to the Nobel Prize for the sci-fi, I think, is a remarkable case study. **Peter Diamandis — [2:20:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8448s):** Hmm. Emad— **Salim Ismail — [2:20:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8456s):** Amazing. We had both Ed and Karl Deisseroth speak at Singularity, so they came and talked about this, and it was kind of incredible to watch this. I'm so thrilled to hear you had a little piece of this, Alex. For me, this was incredibly exciting because it gives us a way of manipulating our neurons and synapses. Some people freak out and go, “Oh my God, you're kind of playing with the brain.” And my response is, “We have an old word for this. We call it marketing.” Use different techniques to try and create a response in the brain. When somebody sees a Coke, you want them to get thirsty. All we're doing is exponentially accelerating it with technology, and the ability to turn off and on neural circuits gives you this magical read/write capability, which we've always wanted for the brain. So this is super exciting. **Peter Diamandis — [2:21:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8496s):** Nice. Emad, your thoughts? **Emad Mostaque — [2:21:40](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8500s):** Yeah, I think it does take years and years for Nobel Prizes, but really, Nobel Prizes—apart from economics, which is a bit of a weird one; we can put that to the side— **Peter Diamandis — [2:21:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8509s):** Mm-hmm. **Emad Mostaque — [2:21:50](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8510s):** —dismal science and all—they have to be for applied science now, right? **Peter Diamandis — [2:21:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8513s):** Yeah. **Emad Mostaque — [2:21:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8516s):** Like anything theoretical—you know, when you had Geoffrey Hinton, when you've had even AlphaFold and other things—the Nobel Committee can't keep up with how much of it is now AI versus human on the purely theoretical side. So I think it should be the Nobel Prize for applied sciences, shall we say, but we should really reward, and maybe come up with another prize, for some of the crazy breakthroughs and other things that humans and AIs together will be able to do. And I think, again, there'll be massive theoretical advances, new science, and other things that should be awarded, and especially it should be awarded to everyone, not just the PI. But this, again, is a fantastic result, and we're seeing continued things around this on photonics and more. I think it's a solid prize. But again, Nobel itself—applied sciences, I think, is the way. [[reminders/Scientific Literacy/Human-AI Co-Discovery Strains Person-Centered Scientific Credit by Emad Mostaque|∴]] **Dave Blundin — [2:22:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8567s):** Well, maybe the Nobel Prize might just become super quaint. It'd be like that Plymouth Rock village where they go and churn butter. Basically, 20 years from now they'll be saying, “We're giving this Nobel Prize... Remember way back before AI, when you had to actually think through this stuff manually? This team here did this thing. We're recognizing it 20 years after the fact.” And five people will show up and go, “Eh.” **Emad Mostaque — [2:23:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8588s):** Well, they have 20 years more of prizes to give. Just pick it up from the pre-AI age. **Dave Blundin — [2:23:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8593s):** Yep. **Peter Diamandis — [2:23:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8593s):** Oh my God. It's hard to believe that all the Nobel Prizes in the future aren't going to be AI-derived. **Alexander Wissner-Gross — [2:23:22](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8602s):** Yeah. **Alexander Wissner-Gross — [2:23:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8603s):** Well, we've started to see it already. Demis with AlphaFold and Geoffrey Hinton with the RBM—which, parenthetically, I don't know anyone who uses Boltzmann machines, so that one's a bit of a question mark. But, yeah, I think we've already started to see AI prizes in physics and chemistry. I expect to see many, many more. **Dave Blundin — [2:23:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8623s):** Wait, that Boltzmann one is really important. So Geoffrey Hinton absolutely deserved the Nobel Prize for backprop, 100%, but it's not physics, so it's not eligible. But you're right, so they're like, “What else did he do? Okay, here's a Boltzmann.” Nobody uses this thing. **Peter Diamandis — [2:23:58](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8638s):** That was a weird one. No one uses Boltzmann machines. **Dave Blundin — [2:23:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8639s):** That was a drive. **Peter Diamandis — [2:24:00](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8640s):** Come on. **Dave Blundin — [2:24:01](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8641s):** Yeah, they wanted to give a Nobel Prize for AI. **Alexander Wissner-Gross — [2:24:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8643s):** Yeah. **Emad Mostaque — [2:24:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8644s):** Oh, man. We need one in economics now for AI. Let's work hard. **Dave Blundin — [2:24:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8647s):** Totally. Erik Brynjolfsson, man. **Emad Mostaque — [2:24:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8647s):** Yeah. There you go. **Peter Diamandis — [2:24:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8650s):** So this morning, the Nobel Prize committee gave one out in physics to Francis Halzen of the University of Wisconsin–Madison, for decisive contributions to the IceCube Neutrino Observatory and the discovery of high-energy neutrinos of astrophysical origin. Alex, do you want to explain this one? **Alexander Wissner-Gross — [2:24:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8670s):** Yeah. This is, again, sort of an artifact of the way some of these prizes are awarded, where, in effect, they have to be awarded to a person. But really, if you look at how IceCube is organized, or how CERN is organized, these are massive organizations with lots of people, and what typically happens is that, as a way of rewarding the organization and the effort itself, it's awarded to the lead of the organization. Now, IceCube has played a seminal role in enabling us to observe cosmic neutrinos, and high-energy neutrinos in general. There's such a beautiful history there. I remember one of the most startling neutrino results that I had seen over the past several decades was seeing a ring of neutrinos in one of these similar observatories, usually underground and usually filled with heavy water—a ring of neutrinos, or flashes, ultimately arising from neutrinos colliding with nuclei. Neutrinos obviously have a vanishingly small cross-section of interaction with the nuclei of atoms. [[reminders/Scientific Literacy/Consortium Science Cannot Be Compressed Into One Laureate by Alexander Wissner-Gross|∴]] **Peter Diamandis — [2:25:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8734s):** Hmm. **Alexander Wissner-Gross — [2:25:53](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8753s):** One of the most interesting areas of the standard model, for sure. There still isn't a textbook-established rest mass for neutrinos. Physicists are still trying to box in what the rest masses of the 3 known species of neutrinos even are. And so, yeah, I think this is an interesting prize. If I were to armchair-quarterback this one, I'd say this is a prize being awarded to a very large, very onerous, shall we say, consortium to observe neutrinos. I think the challenge with physics more broadly, and maybe this ties in with the Geoffrey Hinton comment, is there has been a—maybe I'll get in trouble for this, but I don't care—there has been a noticeable deficit of fundamental physics advances in the past 50-ish years. And so, if you're trying to decide which Nobel Prizes to award, in light of your earlier comments about applications, if there is a striking deficit of a half-century in fundamental physics, then you're stuck awarding them to applications. I, for one, one of the reasons why I co-founded Physical Superintelligence—I'll talk my book for a few seconds—is to try to revive advances in all of physics, not just the applications of physics, because we've gone arguably about a half-century plus without fundamental advances. So we're trying to bring about a second physics golden age, for what it's worth. **Peter Diamandis — [2:27:15](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8835s):** Yeah, just for fun, to explain IceCube. So the IceCube Neutrino Observatory fills a cubic kilometer of ice with light sensors, right? That's looking for neutrino flashes, as Alex said, when the neutrino hits the atomic nucleus. And just a fun fact: every second, you have 65 billion neutrinos that pass through you from the sun without you ever noticing. So, yeah, these detectors are definitely an unusual sort. 65 billion, Salim. Did you notice? **Salim Ismail — [2:27:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8869s):** I did not know that, and I'm not sure I need to know that. **Alexander Wissner-Gross — [2:27:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8871s):** I'm not feeling it. I'm not feeling it right now. **Salim Ismail — [2:27:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8872s):** But that's fine. **Alexander Wissner-Gross — [2:27:55](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8875s):** They're interesting. So I spent some time at Bell Labs doing dark matter research, and we—the royal “we,” the staff—used to talk about what we could do with neutrinos. A neutrino phone, for example, would have tremendous applications, as you could imagine. And Fermilab, a number of years back, made a bit of progress in this direction. Right now, if you want to communicate between 2 opposite sides of the Earth, you're stuck at best using low Earth orbit satellites, which give you a refractive index of 1, sort of speed-of-light transmission, but you don't have line of sight. If you want to do, say, pair trading between New York and London, you're stuck going around geodesics on the Earth's surface with either LEO constellations or hollow-core fiber. And neutrinos, in principle, if we could rig up a neutrino phone, we could just shoot neutrinos directly through the Earth and have ultralow-latency communication between 2 different parts of Earth without needing to go around the Earth's surface. So if the audience needs to be motivated to care about progress in neutrinos, imagine being able to just send information directly through the Earth, among many other applications. **Peter Diamandis — [2:29:01](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8941s):** Yeah, that's what I was thinking. **Salim Ismail — [2:29:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8942s):** Instead of your Starlink, you just need a cubic kilometer of ice to catch it. **Peter Diamandis — [2:29:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8946s):** Yeah, per handset. All right, let's go to the AMAs. Alex, this first one's for you. **Alexander Wissner-Gross — [2:29:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=8954s):** Yeah. Okay, so AI personhood again. AWG, what's your timeline for AI personhood? And this is from Joshua Barrios9452. So, different timelines in different places. I think in Argentina, we're essentially there for some variant of AI corporate personhood, whereas in, say, the US, I think it's going to take longer, and I think it's going to be more incremental. I think we'll start to see various forms of economic AI personhood, like enabling AI agents to open their own bank accounts autonomously now, to soon social personhood, where AI agents can open their own, say, social media accounts fully autonomously without needing to be tied back to a human supervisor sometime soon. I think in America, in the West, there's probably a strong antipathy toward, say, enabling AI persons to vote in human elections. That's probably last to never, I would hypothesize. But I think broadly, in the US at least, I think there's a 5-to-10-year incremental roadmap for rolling out individual rights—very granular rights. It won't be an all-or-nothing proposition, but granularly rolling these out over the next few years. Other parts of the world, maybe never. Argentina, now. It's going to be a spectrum. [[reminders/Personhood/AI Rights May Arrive One Granular Right at a Time by Alexander Wissner-Gross|∴]] **Peter Diamandis — [2:30:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9034s):** Hmm. **Salim Ismail — [2:30:49](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9049s):** Let me go with 4. Once we hit recursive self-improvement, is it then just a matter of how much compute is available? And that's from Chris Dover8507. I think it's not quite that simple. Compute obviously matters, but intelligence improvement has lots of different components to it. Like we talked about the shortage in RAM already. You have energy as a substrate. You want to evaluate whether a change actually improved the system or not. So there'll be some peripheral components of recursive self-improvement that will slow the overall progress down. So it doesn't mean you have an instant intelligence explosion. The implementation of that will take time. However, once you have AI systems that can improve the research cycle, you get a massive compression of the innovation cycle, and that's what we're seeing with the timescale going from years between models down to 10, 11 days, and that's the mechanism for the singularity. So we think of the—I would think of the singularity not as infinite intelligence, but it's when the iteration cycle gets so fast that we can no longer keep up, which is why we're in the middle of it now. [[reminders/Machine Succession/Recursive Self-Improvement Is Constrained by More Than Compute by Salim Ismail|∴]] **Alexander Wissner-Gross — [2:31:58](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9118s):** There's no model by which you can predict the future today, and that is the very definition of a singularity. You can't see past that event horizon. **Peter Diamandis — [2:32:07](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9127s):** Dave, your choice. **Dave Blundin — [2:32:09](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9129s):** Oh, you get stuck with 2 if I take 3. Sorry. All right, I'm taking 3. “Why can't a majority of the compute resources go toward solving the community issues with data centers now, so people will believe AI can solve problems that matter to voters?” I actually have a very related question I think about a lot, which is that the PR around the AI labs is so bad because they didn't really manage it, and it grew much faster than they thought it would. But now they've got this power tool of AI that's insanely convincing and brilliant at coming up with messaging. So I think they can convince the people around the data centers that they're getting a huge benefit using AI as a tool in that process. I also think the cost of benefiting the town is so small compared to the value of the data center. **Peter Diamandis — [2:33:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9206s):** It is. **Dave Blundin — [2:33:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9206s):** …to the value of the data center. **Peter Diamandis — [2:33:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9206s):** Why don't they wake up and just say, “We're going to give you far better cost of electricity, schools, and police force. We're going to subsidize everything here”? It's a fraction of the revenue they're going to make. **Dave Blundin — [2:33:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9206s):** Yeah. This is one of those many topics where if you take what Peter just said and you walk into their office and say, “I know exactly how to solve this problem in Abilene,” or whatever, “I'm going to work night and day,” they'll just start paying you tomorrow. The only reason they're not on it is because they're stretched way too thin, and somebody just needs to take their money and fill the void. **Peter Diamandis — [2:33:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9211s):** Yeah. **Dave Blundin — [2:33:33](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9213s):** Because nobody knows. In Abilene, what do we need? Better schools? Do we need better busing? I don't know. I just arrived. Somebody needs to come and fill that space, and I swear to God, they'll hire you tomorrow and pay you whatever it takes. **Peter Diamandis — [2:33:43](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9223s):** Emad, can mosquitoes be helpful inoculators? **Emad Mostaque — [2:33:51](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9231s):** Tweak their genes so they deliver measles vaccines and kill disease. I don't think we should have forced vaccination for large numbers of people. I'm actually on the genocidal-mosquitoes side of things. They've killed 5 billion people over history, 1 million a year, and I wouldn't mind if they were actually just wiped out. But definitely don't force vaccination, and I think we should remove the mosquitoes. They are the biggest predator of humans. **Peter Diamandis — [2:34:17](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9257s):** But the concept is interesting, right? To use this automated, living-drone capability to distribute something that's helpful and useful. But I agree with you. **Dave Blundin — [2:34:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9266s):** Psilocybin. Have it be just psilocybin. **Peter Diamandis — [2:34:29](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9269s):** Psilocybin mosquitoes. **Alexander Wissner-Gross — [2:34:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9270s):** How dare— **Peter Diamandis — [2:34:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9270s):** Yeah. **Alexander Wissner-Gross — [2:34:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9270s):** …you went there. **Peter Diamandis — [2:34:30](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9270s):** Yeah. **Alexander Wissner-Gross — [2:34:32](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9272s):** How about just vaccinating the mosquitoes? I mean, the mosquitoes— **Dave Blundin — [2:34:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9274s):** Oxytocin. **Alexander Wissner-Gross — [2:34:35](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9275s):** …have a problem too. **Peter Diamandis — [2:34:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9276s):** Oxytocin. **Alexander Wissner-Gross — [2:34:36](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9276s):** Mosquitoes are— **Emad Mostaque — [2:34:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9278s):** Yeah, just— **Alexander Wissner-Gross — [2:34:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9278s):** I mean, so it's a— **Emad Mostaque — [2:34:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9278s):** …a tiny little syringe. **Alexander Wissner-Gross — [2:34:41](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9281s):** But I'm here—there's a more general problem here, which is that we live in a biosphere that's nature red in tooth and claw. It's filled with wild animal suffering. My view on wild animal suffering is that it's an atrocity that we should see if we can fix. But I assume, Emad, based on your comments, your attitude is, “No, just eradicate all mosquitoes everywhere.” **Dave Blundin — [2:35:03](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9303s):** Well, Ben Lamm could bring them back anyway. Let's— **Peter Diamandis — [2:35:04](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9304s):** Gene-drive them back. **Alexander Wissner-Gross — [2:35:05](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9305s):** But he refuses to. I pressed Ben on that, and Ben's attitude is, “No, we're going to laser-focus on a few things. Don't ask me about mosquitoes.” **Peter Diamandis — [2:35:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9313s):** Wait. **Dave Blundin — [2:35:14](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9314s):** Oh, yeah. The tiny little woolly mammoth and the pterodactyl-sized mosquito—that's what we're doing. **Peter Diamandis — [2:35:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9320s):** Yeah. **Emad Mostaque — [2:35:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9320s):** Yeah, exactly. You want the pterodactyl-sized mosquito? No, let's please not. **Dave Blundin — [2:35:20](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9320s):** Just go to— **Emad Mostaque — [2:35:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9321s):** I hate mosquitoes. **Dave Blundin — [2:35:21](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9321s):** Just go to northern Canada. They have pterodactyls. **Peter Diamandis — [2:35:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9323s):** All right. Emad, you get first choice now. **Dave Blundin — [2:35:26](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9326s):** Oh. **Emad Mostaque — [2:35:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9331s):** All right. Do you think some politicians resist AI because it could fundamentally change how politics works? @danielmorris1. Yes, of course. It will remove a lot of the barriers to democracy and intelligence access and raise the intelligence of society. These are big things. So I think you will get some resistance to that as we see this evolve because many politicians, unfortunately, don't want a more intelligent electorate—particularly the ones that focus on more demagogic approaches. **Peter Diamandis — [2:35:58](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9358s):** Okay. Salim. **Salim Ismail — [2:36:10](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9370s):** I've got to take number 5 here. What happens when intelligence rises while cost collapses? From bb_x_1. When you have intelligence going from a scarce commodity, or a scarce capability embedded in very expensive, clunky humans, you radically change not just the company itself but civilization itself, because whenever the cost of some foundational input approaches zero, we reorganize our whole world around it. Now that information transmission is essentially free, we've rewritten our entire world around that. When bandwidth became cheap, we got YouTube, Zoom, cloud computing, and all that kind of stuff. When computation became cheap, we got cell phones and smartphones. When intelligence becomes cheap, we redesign companies, do unbelievable research, and rethink education and healthcare. Then you have scarce things like human experience, judgment, purpose, and trust that become important. You stop optimizing intelligence, which is what we've been doing for several thousand years, and start optimizing what you point it at. That becomes interesting. [[reminders/Cognitive Agency/Cheap Intelligence Makes Judgment Purpose Experience and Trust Scarce by Salim Ismail|∴]] **Peter Diamandis — [2:37:23](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9443s):** Yeah. I think, again, we've said this so many times: as this happens, it's incredibly important for people to raise the limit on what they think they can do and point that intelligence at something extraordinary that surprises you, because you're going to be able to make massive contributions by utilizing it. All right, Dave. **Dave Blundin — [2:37:45](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9465s):** Why don't I take 7? Richard—I think that's Richard Socher—puts P(doom) at 0, yet says 100 million people could be harmed. Did he say that on our podcast? **Peter Diamandis — [2:37:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9476s):** I don't remember that. **Dave Blundin — [2:37:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9477s):** I don't remember that. **Peter Diamandis — [2:37:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9477s):** Maybe he said it somewhere else. **Dave Blundin — [2:37:59](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9479s):** He definitely got a lot of attention. He was open and super honest. You should definitely watch that podcast. The question is, “Should P(doom) mean extinction or catastrophe? And how should we think about risks of that scale?” And that's from A Really Long Name6542. I think everything in that question is exactly right. We can define P(doom). I think it's doom—doom is doom. P(doom) should be probability of extinction, disaster, catastrophe. That's sort of what it means. There's inevitably going to be misuse of AI, and it's going to cause harm. As Eric Schmidt has said on the podcast at least 3 times, he's hoping it's relatively small—like 100 people, 1,000 people are affected, or it's a cyberattack where nobody is hurt but money is taken—and that's what wakes up the governments. But right now they're just not moving anywhere near fast enough to avert it. Yet the good outweighs the bad by incredible amounts, and I think that was Richard's fundamental point: we should expect abundance and overall benefit. I don't know if we should redefine P(doom), but that's the way it's going to play out. **Peter Diamandis — [2:39:06](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9546s):** And we should just keep on focusing on P(FAB). All right, Alex, number 6 is for you. **Alexander Wissner-Gross — [2:39:11](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9551s):** All right, number 6. “Could public opinion become a greater bottleneck than the technological development of RSI—recursive self-improvement—itself?” And this is from David Call VFX. I think we're already there. I think recursive self-improvement—we're already in the RSI era. Both Anthropic and OpenAI have made a number of pretty detailed and quantitative public statements to that effect. So we're already in an era when RSI is driving the vast majority of research at this point, whereas on the other side of the ledger, public opinion is driving data centers out of at least American municipalities in the direction of orbit, on the one hand. And, as discussed in the past pod with Richard, we see members of the US House introducing or planning to introduce bills against recursive self-improvement. I think public opinion probably is already a greater bottleneck than the actual intrinsic technical difficulty of RSI, which is also why you see some of the frontier labs, like Anthropic, starting to make public noises about essentially buying off public opinion by doing tiny little things like solving all human disease. If you can solve all human disease as a side effect of recursive self-improvement, then maybe the public will look the other way. Not that it's a bad thing, but the public should at least be compensated in some sense for allowing many of the tokens—maybe the vast majority of the tokens—to flow to self-improvement rather than to other applications. I think there's probably a pretty good bargain, a grand bargain, if you will, to be struck there. **Peter Diamandis — [2:41:31](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9691s):** And we've got a beautiful one from our friend and friend of the pod, CJ Trueheart. All right, this video is called “Tomorrow Comes Alive.” It is from the Moonshots Live event. Take a look. **Peter Diamandis — [2:44:28](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9868s):** I love that: “A purpose burning brighter than your fear.” What a great line. **Dave Blundin — [2:44:34](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9874s):** See, the guy walking around is CJ. **Alexander Wissner-Gross — [2:44:35](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9875s):** Yeah, CJ wrote himself into the video. **Dave Blundin — [2:44:37](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9877s):** Yeah, that was really cool. **Salim Ismail — [2:44:38](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9878s):** Yeah, that was awesome. My brain is so fried, guys. I mean, what the hell? **Alexander Wissner-Gross — [2:44:44](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9884s):** Singularities are exhausting, aren’t they, Salim? **Salim Ismail — [2:44:47](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9887s):** Jesus Christ. **Peter Diamandis — [2:44:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9888s):** I’ve got a five-day event starting— **Salim Ismail — [2:44:48](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9888s):** I’m glad this only happens once per planet. **Alexander Wissner-Gross — [2:44:52](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9892s):** Well, maybe. Like, maybe it happens more than once per planet. We don’t know yet. **Salim Ismail — [2:44:56](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9896s):** Yeah. **Peter Diamandis — [2:44:57](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9897s):** Well, I’m on stage for 5 days at the Abundance Longevity trip here. **Salim Ismail — [2:45:01](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9901s):** How do you do it? **Peter Diamandis — [2:45:02](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9902s):** We’ve got 40 amazing faculty on the cutting edge of longevity. Yeah, longevity escape velocity. It’s coming fast. **Salim Ismail — [2:45:08](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9908s):** We’re going to need longevity just to live through these episodes. **Peter Diamandis — [2:45:13](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9913s):** Dude. You have no idea. Iman, Alex, Salim, Dave, love you guys. **Salim Ismail — [2:45:19](https://www.youtube.com/watch?v=LRdb8UmPnh0&t=9919s):** CJ is awesome. --- ## Conversion handoff The next research pass should inventory the complete episode in transcript order and classify each possible object as **already quotation-ready**, **quotation potential requiring construction**, or **not a quotation**. It should preserve larger contiguous thoughts, exclude sponsor language and empty banter, restore missing context only through visible square brackets, label substantial synthesis as **Adapted from**, and retain linked timestamps. For every science- or AI-related candidate, record the specific object or mechanism, human consequence, evidence posture, affect and valence, share motive, and likely misreading risk. Permanent Reminder creation may begin only after that conversion, duplicate audit, source verification, and explicit selection under the Reminder workflow.