# Richard Socher: ASI and RSI Timeline, Khanna Moves to Ban Recursive AI — Reconciled Transcript Source This is the prepared transcript source for a future complete standalone-quotation conversion of [*Richard Socher: ASI and RSI Timeline, Khanna Moves to Ban Recursive AI, and Tavus’ Turing Test*](https://www.youtube.com/watch?v=Blyb1D927pM). It preserves the discussion in broadcast order under the publisher’s chapter structure without prematurely atomizing every sentence into a quotation. The later quotation edition should follow the larger-intellectual-unit policy used in [[research/Jensen Pushes Back on Doomers Moonshots Live]]: a mechanism, qualification, example, and conclusion remain together when they form one argumentative movement. **Source record.** The user-supplied document contains the YouTube auto-generated transcript, publisher description, participant list, and chapter map. A second complete transcription was produced from the same YouTube recording with ElevenLabs Scribe v2 using English-language recognition, speaker diarization, word-level timestamps, audio-event tagging, and verbatim mode. ElevenLabs returned 54,862 timed tokens across 8,749.302 seconds of audio with language probability 1.0. This prepared source uses the ElevenLabs speaker turns and timestamps; the YouTube transcript remains an independent comparison layer, and the recording remains the final authority for consequential wording. **Speaker reconciliation.** ElevenLabs separated nine acoustic classes. Repeated direct address, self-identification, the opening introductions, and argument continuity map the five discussion voices as follows: `speaker_0` is **Peter Diamandis**, `speaker_1` is **Richard Socher**, `speaker_2` is **Dave Blundin**, `speaker_3` is **Alexander Wissner-Gross**, and `speaker_5` is **Salim Ismail**. Four non-panel classes are excluded from the discussion transcript: production or promotional interjection, Function Health sponsor read, Blitzy sponsor read, closing generated song. **Editorial status.** The speaker turns below are a reconciled transcription source, not yet the final quotation conversion and not a Reminder collection. Filler, false starts, repetitions, antecedents, speaker interruptions, proper nouns, technical vocabulary, and factual claims have not yet received the full object-by-object editorial treatment used in the two model research files. No line below should be promoted as a verbatim Reminder until it has been checked against the recording, made independently intelligible with visible bracket repair where necessary, and assigned either ordinary attribution or an explicit **Adapted from** label. **Preservation boundary.** This source retains all turns assigned to the five panelists, including questions and brief reactions, so the later conversion can decide intellectual boundaries from complete conversational context. It does not convert caption fragments into quotations, silently join speakers, or discard short language before its relationship to adjacent turns has been assessed. **Conversion target.** The next pass will turn this evidence layer into a complete standalone-quotation research edition designed for later Reminder promotion. The preferred objects are cutting-edge, groundbreaking, distributable statements that preserve the state of the art, expose an operative mechanism, articulate a consequential prediction, or give a reader a durable model of what is changing. The conversion must represent the full intellectual range of the broadcast rather than overharvesting only recursive self-improvement, AI regulation, or Richard Socher's answers. **Granularity rule.** The unit of conversion is an independently viable intellectual movement, not a caption, pause, sentence, or speaker-turn boundary. A mechanism, qualification, example, forecast, and conclusion should remain together when separating them would produce reminders that no longer explain what they concern or why they matter. A long source passage may later support several Reminders only when each proposed division independently names its subject, preserves the speaker's epistemic posture, and reaches its own consequence. Brief host questions and interjections may supply context for understanding an answer, but their words must never be silently placed inside another speaker's quotation; missing context should be restored transparently with square brackets or, when substantial construction is required, through an explicit **Adapted from** object. **Reminder-readiness boundary.** The quotation conversion is not itself authorization to create a permanent Reminder. A converted object becomes Reminder-grade only when it can be distributed without the episode title, chapter heading, preceding exchange, or explanatory caption; identifies the relevant technology, institution, scientific object, or predicted event; makes clear whether it describes demonstrated capability, reported result, interpretation, scenario, or forecast; and remains interesting enough that a person would share it to teach, warn, predict, display the inside track, express grounded optimism, or preserve a claim for later vindication. Factual predicates that would become assertions on a poster must be checked against the recording and, where consequential, against a primary source before promotion. ## Quotation-conversion coverage map The conversion should preserve the strongest independently viable objects in every major domain below. The map is a coverage obligation, not a quota: weak material should not be promoted merely to fill a category, but no domain should disappear because another topic supplies more dramatic language. - **Recursive self-improvement and ASI timelines:** weak and strong forms of RSI, architectural self-improvement, world knowledge versus reasoning kernels, the relationship between RSI and ASI, and competing time horizons. - **AI-driven scientific discovery:** *The Eureka Machine*, abstraction in science, research coordination, hypothesis-to-experiment compression, mathematical and scientific automation, scientific labor, and the prospect of radically accelerated discovery. - **Brain decoding and brain-computer interfaces:** perception versus thought, decoding limits, neural interfaces, cognitive communication, new training substrates, and the distinction between present demonstrations and extrapolated capability. - **Engineering biology:** gene editing, mosquito interventions, programmable organisms, biological control systems, laboratory automation, medical consequence, and the evidentiary boundary between a demonstrated result and a forecast. - **AI avatars and the Turing boundary:** Tavus, synthetic presence, conversational embodiment, tests of social realism, identity, persuasion, continuity of personality, and what people actually require before treating an artificial interlocutor as socially real. - **AI risk, alignment, and technological panic:** realistic threat vectors, misuse, cybersecurity, biological risk, internet literacy, reward hacking, constitutional AI, liability, enforceability, defensive responses, and the difference between regulating harmful applications and attempting to suppress general capability. - **The Human Control Over AI Act and recursive-AI prohibition:** Ro Khanna's proposal, containment and shutdown requirements, public support, criminal penalties, the possibility of an AI surveillance state, enforcement limits, corporate chilling effects, historical technology panics, and the political incentives behind categorical bans. - **Frontier-model competition:** Gemini 4 Argon, GPT-6.1 Sol, Opus and Sonnet 5.5, benchmark interpretation, hallucination reduction, long-horizon reasoning, model specialization, open models, laboratory competition, and the danger of mistaking one benchmark for general superiority. - **Compute economics and intelligence price-performance:** internal compute scarcity, cost-performance frontiers, inference economics, falling intelligence prices, data-center and chip constraints, resource allocation, enterprise adoption, and the strategic implications of cheaper cognition. - **Defense and warfare:** Project Meridian, Elon Musk, Palmer Luckey, autonomous and software-defined warfare, deterrence, procurement, military transformation, and the distinction between an announced program and a demonstrated capability. - **Work, investment, and economic transformation:** job displacement, productivity, AI orchestration, entrepreneurship, investment signals, enterprise restructuring, comparative advantage, and how people and institutions may capture value during the transition. - **Attention, agency, and human adaptation:** the attention economy, education, scientific careers, public understanding, individual initiative, optimism, and the behavioral changes required to benefit from accelerating capability. - **Predictions worth preserving:** concrete forecasts about timing, capability, cost, adoption, scientific breakthroughs, regulation, labor, security, and social response whose conditions are explicit enough to be judged later rather than merely remembered as dramatic rhetoric. ## Consolidation and source-fidelity gates Before Reminder selection begins, the converted edition must pass four gates. First, adjacent language must be reviewed for over-atomization, with separated fragments recombined whenever the full movement produces the stronger and more intelligible object. Second, every quotation must survive removal of the question that prompted it, with missing subjects and objects restored visibly rather than guessed. Third, technical names, model names, benchmarks, legislation, organizations, people, numerical claims, and time horizons must be reconciled against the recording and available primary material. Fourth, each object must be labeled honestly as source-faithful or **Adapted from**, so later Reminder pages can preserve both readability and chain of custody. ## Cold Open: Recursive Self-Improvement, ASI, and the Week Ahead **Peter Diamandis — [00:00](https://www.youtube.com/watch?v=Blyb1D927pM&t=0s):** Uh, Richard, I'm, I'm curious, where are we at the moment with RSI? **Richard Socher — [00:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=5s):** In various weak forms, we already have RSI. We're not quite there yet, but we're very close. **Peter Diamandis — [00:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=10s):** When do you believe we reach ASI? **Richard Socher — [00:14](https://www.youtube.com/watch?v=Blyb1D927pM&t=14s):** I think it will take us probably several decades. **Peter Diamandis — [00:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=18s):** On Terminal Bench 4.0, Sonnet 5.5 jumped from 10% to 70%. Pretty extraordinary. **Dave Blundin — [00:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=24s):** My theory would be that they're trying to compete with, you know, China, which is about three months behind, and fill that gap before a lot of enterprises go to, to open source models. **Alexander Wissner-Gross — [00:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=34s):** It's not at all obvious to me why anyone should be using Sonnet 5.5 over Opus 5.5, unless you have some token or latency or other consideration. I do not plan to use Sonnet 5.5. **Peter Diamandis — [00:45](https://www.youtube.com/watch?v=Blyb1D927pM&t=45s):** The Defense Secretary announced Project Meridian. It's a new Pentagon effort on the future of warfare. It's co-led by Elon Musk and Palmer Luckey. **Alexander Wissner-Gross — [00:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=54s):** This, I think, is a transformative moment. This is a major, major step forward. **Dave Blundin — [00:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=59s):** The big question's gonna be- **Peter Diamandis — [01:14](https://www.youtube.com/watch?v=Blyb1D927pM&t=74s):** Welcome to Moonshots, everyone, your number one podcast in helping understand the singularity, what's going on, what does it mean to you and your family, and what the breaking news is. We publish this twice a week to help you keep up with the extraordinary rate of change. With me today is the Fantastic Four, Alex Weiszer-Grose, our in-house ASI- **Peter Diamandis — [01:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=95s):** Dave Blundin, our impresario of AI investing; Salim Ismail, our globetrotter, who's home today. Amazing, Salim. **Dave Blundin — [01:43](https://www.youtube.com/watch?v=Blyb1D927pM&t=103s):** Woo-hoo. **Peter Diamandis — [01:43](https://www.youtube.com/watch?v=Blyb1D927pM&t=103s):** And the and the father of the organizational singularity, I'm Peter Diamandis, your host and data-driven optimist. Our mission here, keep you optimistic about the future. Uh, and here's the numbers from this week. In seven days, Google shipped Gemini 4 argon, Anthropic shipped Sonnet 5.5, OpenAI shipped GPT 6.1, Saul, and the frontier intelligence is gotten cheaper by threefold. SpaceX launched the next batch of astronauts to the ISS, and also launched Google's TPUs into orbit for Project Suncatcher. Uh, if you're new to the Moonshots podcast, please hit subscribe. We publish twice a week, and you don't wanna miss any of this news during this hypersonic tsunami. Our moonshot here is to 20X our subscriber base to get to 10 million to help spread the word of optimism and the extraordinary future that we're building. Today on Moonshots, we're joined by one of the architects of modern AI, Richard Socher. Uh, born in Germany and trained at Stanford, Richard is among the world's most cited natural language processing researchers, a pioneer in deep learning and prompt engineering, and a serial founder who's repeatedly turned frontier research into category-defining companies. First, he founded Metamind, uh, which was acquired by Salesforce, where he then became the chief scientist leading AI research for friend of the pod, Marc Benioff. Next, Richard founded You.com, recognized by Time and the World Economic Forum, and now valued at north of $1.5 billion, as well as his fund, Aox- A- uh, AI-X Ventures. Today, Richard is co-founder and CEO of Recursive, perhaps, uh, the biggest moonshot he's ever taken, uh, focused on recursively self-improving super intelligence. Richard, congrats on a $670 million fundraise from Google Ventures, Greycroft, NVIDIA, and AMD, and $410 million in compute from AWS. Personal disclosure, I'm very proud to be a, a seed investor in Recursive, and of course, Richard, uh, important to mention, your new book just got released, The Eureka Machine: Why AI Is the Key to Unlocking a New Era of Scientific Discoveries. Welcome to the pod, Richard. **Richard Socher — [03:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=239s):** So great to be back. Uh- **Dave Blundin — [04:00](https://www.youtube.com/watch?v=Blyb1D927pM&t=240s):** Hey, and let me just add- **Richard Socher — [04:01](https://www.youtube.com/watch?v=Blyb1D927pM&t=241s):** Always love- **Dave Blundin — [04:01](https://www.youtube.com/watch?v=Blyb1D927pM&t=241s):** ... a, a truly, truly awesome guy. A lot of people who listen to the pod are worried about the, the ethics and the, the risks and everything, but boy, if you want a, a person to conquer recursive self-improvement that you can like and trust, Richard is the man. **Peter Diamandis — [04:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=256s):** Yeah. **Richard Socher — [04:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=257s):** Thank you so much. So great to be back here. I love your guys' constructive optimism. **Peter Diamandis — [04:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=261s):** I have a bold- **Richard Socher — [04:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=261s):** So important in the world right now **Dave Blundin — [04:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=262s):** I have a, I have a bold prediction for this episode. **Peter Diamandis — [04:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=264s):** What's that? **Dave Blundin — [04:25](https://www.youtube.com/watch?v=Blyb1D927pM&t=265s):** This one episode will prove our thesis for this whole podcast more than any other episode we ever record. **Peter Diamandis — [04:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=272s):** Awesome. **Richard Socher — [04:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=272s):** Wow. Let's go. **Dave Blundin — [04:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=274s):** Let's go. **Peter Diamandis — [04:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=274s):** And, uh, uh, Alex, do you know Richard? **Alexander Wissner-Gross — [04:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=277s):** Yeah, R- Richard, you and I have chatted, uh, quite a bit. I don't think- **Richard Socher — [04:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=280s):** Yep **Alexander Wissner-Gross — [04:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=281s):** ... we've actually ever met in person, though. **Richard Socher — [04:43](https://www.youtube.com/watch?v=Blyb1D927pM&t=283s):** Really? We have not, and it's the first time on a podcast together. It's gonna be awesome. **Alexander Wissner-Gross — [04:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=286s):** Amazing. Well, let's make history. ## The Eureka Machine and AI-Driven Scientific Discovery **Peter Diamandis — [04:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=288s):** Yeah, for sure. So, uh, Richard, let's start with your book. Uh, the central claim of The Eureka Machine is that AI will deliver a century of scientific breakthroughs in the next decade. Uh, you know, this maps directly onto what Alex and I wrote in Solve Everything, so we're fans of that prediction. Uh, your thesis in the book is that every stage of scientific process gets connected and transformed simultaneously. Hypothesis, experiment, data, theory. You call it the full-stack AI. Uh, you also say that scientific progress has slowed, and that's been caused not by underfunding, but by fragmentation. So let's start with those two items. Could you take a moment and, you know, talk to us about full-stack AI for science and why you think progress has slowed? **Richard Socher — [05:39](https://www.youtube.com/watch?v=Blyb1D927pM&t=339s):** Yeah. So progress has slowed, uh, largely because we have so many different sub-discipline and, uh, disciplines, and even Stanisław Lem, uh, realized like, wow, that like, if there are only so many people, and we have more and more fragmentation of more and more sub-disciplines and niches, like you can not just do AI, right? You're doing often like optimization, uh, and gradient descent methods, and second order derivative methods, uh, and so on of, uh, these neural networks that is-- which is one category of AI and so on, and the same is true in biology. You study biology, you're gonna be either a cell biologist or a molecular biologist, uh, or you're like in medicine, and like these... Like, there's so much separation that it's hard to weave it all together, and that is the perfect time for AI to come in and help us weave back together all these separate pieces, and also help us understand how these large complex systems actually work. Like, we can't have one model where humans say, "Here are all the things I know about natural language, and now like we can put all these rules together, and then we have a conversation." It required a large neural network with a ton of data, and guess what? We're gonna get a lot of data, uh, about biology, and so we can make more and more disciplines and move them, transfer them from being traditionally natural sciences, where we just try to understand biology and nature, to being programmable engineering sciences. And that, I think, uh, is one of the many different exciting aspects of things to come. And, you know, why, why am I so excited that, that this is happening right now, and, and what are the major ingredients? It's essentially that we now have world knowledge in the form of LMs. We have more and more scientific data that's getting digitized that we can sit on top of. We have better and better simulations of things, and we have robotic process automation that will soon be possible, and on top of that, we're gonna have an agent swarm, and that is how, uh, this, the full scientific stack, uh, can be automated. [[reminders/Scientific Acceleration/AI Can Reassemble the Fragmented Scientific World by Richard Socher|∴]] **Peter Diamandis — [07:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=456s):** Alex? **Alexander Wissner-Gross — [07:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=458s):** Yeah, I can't say I disagree. As Peter, you and I wrote in Solve Everything: Math, Science, and Engineering Are Cooked. I'm curious, Richard, what your latest timelines are. **Richard Socher — [07:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=468s):** I mean, the-- I think every field is going to get to higher and higher levels of abstraction. Computer science has been very good at that, right? No one is programming in zeros and ones anymore. No one-- Like, uh, very few people have to still know C++ and complex pointers and, and memory, uh, architectures, and so on. We can, uh, get larger and larger, and computer science has basically abstracted enough so that it met the rest of humanity with English right? And natural language that now can be used to do computer science. And so I think that, um, gives me great hope, uh, that we can get other fields into similar, uh, stages of abstraction and then all meet in, in natural language to do science. [[reminders/Scientific Acceleration/Science Can Meet in Natural Language by Richard Socher|∴]] **Alexander Wissner-Gross — [08:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=510s):** But to pin you down, I, as I recall from, uh, my understanding of your book, three to five years for the physical sciences? **Richard Socher — [08:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=517s):** You know, like there's no like single sort of this is the threshold, and now we solved all the physical sciences, right? But like multiple different diseases will get cured in the next twelve months. They're gonna be the simpler diseases maybe where there's one gene that needs to be fixed for that disease to be cured. Um, uh, but there are a lot of single gene diseases, uh, that, that are out there in aggregate, right? Like, and so those diseases will get cured, and then we're gonna cure more and more complex diseases. We're gonna develop better and better battery material. So I wouldn't call it like one threshold, and like in three years sort of everything is solved, but like we're gonna solve more and more problems, and it'll just be a question of how much money do you wanna put into compute to then solve which kinds of problems. **Alexander Wissner-Gross — [09:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=559s):** You're, you're talking to the guy, Richard, remember, who argues that the singularity itself is something of an optical illusion, that there is no step function, it's just a time interval. So I'll, I'll try once more. Timelines. **Alexander Wissner-Gross — [09:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=571s):** When, when e- even if it's not a step function, when is the inflection point? When is the, um, fifty percent point crossed in your mind for all of the physical sciences getting solved, all human disease getting solved? Where are those timelines? **Peter Diamandis — [09:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=586s):** A- Alex, if I could just add a, a, a layer on top of that. So we've talked on this pod a lot that math is cooked, right? We're seeing- **Alexander Wissner-Gross — [09:52](https://www.youtube.com/watch?v=Blyb1D927pM&t=592s):** Right **Peter Diamandis — [09:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=593s):** ... millennium prizes all falling. **Alexander Wissner-Gross — [09:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=594s):** Yep. **Peter Diamandis — [09:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=595s):** And the nat- the next natural, uh, if you would, uh, uh, barbecue is likely to be physics. **Alexander Wissner-Gross — [10:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=602s):** Incineration? **Peter Diamandis — [10:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=603s):** Incineration, yes. **Alexander Wissner-Gross — [10:04](https://www.youtube.com/watch?v=Blyb1D927pM&t=604s):** And, and computer science arguably, I mean, presumably this is part of the premise, Richard, of Recursive as well, computer science already cooked, math thoroughly cooked, physics- **Richard Socher — [10:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=612s):** So, you know, in a weird way- **Alexander Wissner-Gross — [10:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=613s):** Yeah **Richard Socher — [10:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=613s):** ... I would, I, I, I would use different metaphors. I think they're flourishing, not getting cooked. I think the weird thing is, you know, some b- some mathematicians recently said, "Oh, this is, this may be bad for the field because like we need to train people." And yes, we need to train people, but imagine a biologist or, or medical researcher saying, "Oh, you know, it's really a bummer for the field that we cured all these diseases for people." **Richard Socher — [10:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=633s):** Like that would be insane, right? Of course, you wanna just move as quickly as you can towards solving these problems. I think maybe the problem of math is that there's many sub-fields in, uh, of math that haven't really had... They were just almost purely beautiful intellectual exercises without real-life impact anymore. But a- as you get close to real-life impact, you should be excited about it, and you think and realize your field is flourishing, not being cooked. Uh, and so to timelines, I think the, the further, the more complex a disease is, the more you n- have to do long-term studies with humans before you're allowed to put a certain drug into humans, the more delays you will have. But like we see now companies, to make this very concrete, biotech companies used to have like one new drug in development, and it took them ten years. They had to go public before they knew the com- the drug was really working. And then after ten years, maybe late stage s- three trials were just not working, and then the company is dead, right? Now, you have the new age of companies. They have five to ten different compounds in, uh, stage three trials after two or three years. And, and so like we see a lot of acceleration in that level. But again, diseases, to really be able to get them into humans at scale in the United States with FDA approvals, there are just some natural delays that will be more like half a decade to a decade. Uh, but, uh, lots of other things in chemistry, in physics, and so on, where we can iterate without having to look at long-term human trials will be even faster. **Alexander Wissner-Gross — [12:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=722s):** And I, the, the, the point is well taken. I, I like your Orwellian turn of phrase. Maybe instead of saying AI is cooking math, I should be using flourishing as a transitive verb and just say AI is flourishing math. **Richard Socher — [12:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=736s):** I love it. **Dave Blundin — [12:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=737s):** Actually, I was with, uh, Sertac Karaman, who runs LIDS at MIT, uh, the night before last. He, he started a, uh, drone company now, an AI drone company, but LIDS is where radar was invented originally during World War II. It's a great lab, very entrepreneurial. But he said all his mathematician friends at MIT, uh, are aware that they're cooked- **Dave Blundin — [12:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=755s):** ... and they're, they're all... Yeah, I was like surprised. **Alexander Wissner-Gross — [12:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=757s):** No, no, no, Dave, Dave, we're, we're going with the Orwellian language now. **Alexander Wissner-Gross — [12:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=760s):** They're, they're not cooked, they're being flourished. **Dave Blundin — [12:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=762s):** He, he-- That's funny 'cause he literally said cooked, but I'll go back to him and tell him, like, "Tell your friends they're flourishing." **Alexander Wissner-Gross — [12:47](https://www.youtube.com/watch?v=Blyb1D927pM&t=767s):** Yeah. T-tell them all- **Richard Socher — [12:47](https://www.youtube.com/watch?v=Blyb1D927pM&t=767s):** No, they're all... Well, you think they're in great shape **Alexander Wissner-Gross — [12:49](https://www.youtube.com/watch?v=Blyb1D927pM&t=769s):** ... that you're being flourished, not to worry. **Richard Socher — [12:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=771s):** If your goal was to prove as many theorems as possible in your lifetime, like now is the time to just grab, like grab as many as you can and, and work with AI to solve them. And then I think the field will change- **Dave Blundin — [13:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=782s):** Yeah **Richard Socher — [13:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=782s):** ... the way computer science has changed in many ways. Uh, and it, it'll be much more about what's the most creative thing when you really understand all the things that are out there, Matt, what kind of new formalisms can you create, new constructs can you create that then would be interesting to be solved, uh, by an AI in collaboration with humans? **Dave Blundin — [13:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=801s):** Well, Sertac was saying the math guys are in great shape because they're so cooked that they're all moving over to AI orchestration, and they're gonna be way ahead of the curve. He, he's actually most worried about the biology professors who are in complete denial, you know, using virtually no AI in their day-to-day activities. And this is one area where, Richard, you have a, such a deep background in all facets of AI, including biology. And I, I feel like we're, we're living exactly parallel lives, except you're 15 years younger than me, so I'm like insanely jealous of, of your life trajectory. But be a serial entrepreneur, then start a, a hugely successful venture fund, then found a foundation model company and, and, you know, we're, we're seeing the world through the exact same lens. But you and Peter have much more biology background. I, I have basically none. Uh, but the biologists are the ones really, really lagging, and I, I think you've got a bunch of investments actually that are, that have done really well in the area, right? **Richard Socher — [14:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=852s):** And I would love to talk about some of those, like ParallelBio, Proxima Labs, Ignota Labs. They're truly exciting companies. I think another field that is even more lacking than biology is economics. **Dave Blundin — [14:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=862s):** Mm. **Richard Socher — [14:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=862s):** Economics literally has these models of like a linear model of economics, right? A one-step economy that's provably correctly taxed, uh, and, and subsidized and things like that. It's just like absurd how slow that field is to adopt AI for making better policy decisions. **Dave Blundin — [14:39](https://www.youtube.com/watch?v=Blyb1D927pM&t=879s):** I, I have to insert something- **Richard Socher — [14:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=880s):** It's just all ideology **Dave Blundin — [14:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=881s):** ... because you opened the door. Erik Brynjolfsson, our very good friend, uh, I did not realize- **Richard Socher — [14:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=886s):** Oh, yeah **Dave Blundin — [14:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=886s):** ... till yesterday that HAI Lab, Hai Lab at Stanford invented the word foundation model. And he sent me the whole- **Richard Socher — [14:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=894s):** He did, yeah. **Dave Blundin — [14:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=894s):** You didn't-- You knew that already? Wow. **Richard Socher — [14:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=896s):** Yeah, of course, like yeah, lots of Stanford friends and Percy Liang and others, uh, worked on foundation models. I love Erik Brynjolfsson. He's actually one of the most interesting economists, I think, uh, doing- **Dave Blundin — [15:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=905s):** Uh **Richard Socher — [15:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=905s):** ... really interesting research right now. He also started WorkHelix, which we're a proud investor in, uh, that brings, uh, un- like understanding of how companies actually adopt AI, uh, and which tasks are getting, you know, helped wi- by, uh, in real rollouts for companies. Uh, yeah, he's a co-founder of that too. Uh- **Dave Blundin — [15:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=921s):** Yeah, that building is just such a great place 'cause you talk about econ people being way off the curve, but he is so ahead of the curve. You walk into the building, and you can just feel it. He's also poaching a ton of talent from MIT to come out to- **Dave Blundin — [15:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=933s):** ... and join his lab. **Alexander Wissner-Gross — [15:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=935s):** Following in his path. **Richard Socher — [15:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=936s):** Sorry about that. **Alexander Wissner-Gross — [15:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=936s):** Uh, Richard, talk about full stack AI as you see it in the scientific method. Um, you know, I'm an investor in a company called Leila out of MIT and Harvard. You know Leila Science as well. Uh, I think I've introduced you to, to Jeff Von Maltzahn there. And you know, they're building a scientific super intelligence that is then running a million square foot of robotic space to sort of like a thousand X the rate of discovery. Um, your thoughts on that? **Richard Socher — [16:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=962s):** I absolutely love it. I think that is, you know, in the Eureka machine, I sort of lay out these four pillars, and they're actually one of the few, um, uh, that, that are really going after this, similar to Periodic Labs, who are doing it more on the physics and chemistry side of things. Uh, and there are several other companies now that I can hopefully soon talk about that are trying to create more data for the bitter lesson to be applicable in biology. And I love, uh, what, you know, the big guys, Eli Lilly and others do, uh, and, and Leila too. I think, you know, the-- ultimately, uh, you can boil down the scientific method to the ideation, implementation, and validation of ideas, and the faster we can close that loop and then put an open-ended process on top of it, that's what open-endedness, uh, what inspired us a lot at Recursive, and we have many of the world's greatest, uh, like researchers in that sub-domain of AI that is still not quite as, as popular as it could and should be. Um, uh, like the more you can have a swarm, uh, innovate in open-ended ways and evolve, uh, and combine interestingly different ideas, the better. But then, of course, in biology, you have to have, uh, like actual, uh, like physical, biological wetware experiments, and so it's really great to see, um, uh, Leila doing that. Um, we're seeing this also, maybe I can talk about this one company that I really love called ParallelBio. They build tiny organoids in a petri dish. [[reminders/Scientific Acceleration/The Scientific Method Can Become an Open-Ended Machine by Richard Socher|∴]] **Alexander Wissner-Gross — [17:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=1046s):** Right. **Richard Socher — [17:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=1047s):** And get this, if you love animals, you too can love AI. Why? Because they got FDA approval to skip animal trials. Turns out we cured most diseases in mice. It's not that helpful. They're very different to people. Uh, but these guys, uh, use pluripotent stem cells to create tiny little organoids of lymph nodes. Lymph nodes, big part of your immune system. Immunotherapy is some of the most exciting therapies to allow your own immune system to attack a cancer and so on, instead of getting crazy chemotherapy and so on. And so they got FDA approval because they showed that when you run experiments in parallel, hence ParallelBio, in these tiny petri dishes, how those organoids react to different drugs and toxicity testing and so on, is actually more predictive of how those drugs will interact in real human bodies. And that is just one of many beautiful examples- [[reminders/Biotechnology/Human Organoids Can Outpredict Animal Trials by Richard Socher|∴]] **Alexander Wissner-Gross — [18:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=1090s):** I, I love that **Richard Socher — [18:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=1091s):** ... of where AI will help. **Alexander Wissner-Gross — [18:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=1092s):** And you can take my stem cell, you can create, take my skin, create an, a, uh, iPSC, uh, cell, grow my own organs, and see how a particular drug would work for me versus a generic individual. Uh, Salim, you wanna jump in? **Salim Ismail — [18:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=1106s):** Uh, yeah, a couple of things. One is, you know, science has always been a coordination problem, right? You're kind of trying to bring... And it's always been very structured into departments and, and journals and all that stuff, and I love what you're doing bringing it together. For me, if I had to summarize what you seem to be doing, is you're collapsing the time between an experiment and a result or a hypothesis and a result. And when you can collapse that, that's domain collapse of the scientific method, and now we're like, anything is possible from there. This is amazing. [[reminders/Scientific Acceleration/AI Compresses the Distance Between Hypothesis and Result by Salim Ismail|∴]] **Peter Diamandis — [18:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=1136s):** Yeah. I, I am gonna push again on what, what Alex said. Physics, when do you, you know, uh, you know, Einstein's theory of relativity comes out, there's not that much progress, uh, you know, since then, a lot of theories. Do you see physics as the next domain that's gonna flourish on the back of math? **Richard Socher — [19:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=1155s):** I think the biggest domain is actually going to be biology. Physics has been interestingly stuck in many ways. Like, they, there's just, um, really powerful ideas like E equals MC squared, so you can get a ton of energy out of potentially a little mass, and then we get nuclear energy out of that. Uh, and so, uh, that then moved into and graduated in some ways into an engineering discipline, uh, which is where you have the real, real-life impact. Um, I think, you know, obviously, like, fusion will be great to finally get figured out, and there's a lot of really cool, like, companies and, and big labs and, you know, tokamaks are ready to balance, uh, like plasma inside a tokamak's like, is a very hard control problem that, where AI is being used. Um, I hope we can eventually make better theories for, uh, you know, quantum gravity and all kinds of other complex issues and, and have, get a better world sort of formula. Um, I-- unfortunately, the dataset collections these days require often like a large hadron colliders and just like billions of dollars, and it's like quite expensive, actually. So in a weird way, biology is a better fit for AI because what calculus did for physics, like understanding really well s- micro, uh, in-individualized separated phenomena, neural nets are great in combining all these little things. Like, we know what a neuron does, we know what one, uh, bacteria does in our microbiome, but then as they all come together and form these really complex interactions, we don't really know anymore how that works, and so that's where AI, I think, will help us more. [[reminders/Biotechnology/Biology May Be a Better Fit for AI Than Physics by Richard Socher|∴]] **Peter Diamandis — [20:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=1248s):** Yeah. Alex? **Alexander Wissner-Gross — [20:50](https://www.youtube.com/watch?v=Blyb1D927pM&t=1250s):** Let, let's talk about biology a bit. So do you think the critical path to solving biology goes through digital twins of cells, or do you have some wildly different theory of the case? **Richard Socher — [21:01](https://www.youtube.com/watch?v=Blyb1D927pM&t=1261s):** Three years ago when I started, uh, the Eureka Machine book, I had this, uh, chapter on the virtual cell, and I was really proud of like trying to lay it all out, and I had to rewrite that whole thing 'cause in the meantime, uh, you know, Mark Zuckerberg Foundation started virtual cell. Like, a lot of people are now working- **Alexander Wissner-Gross — [21:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=1273s):** Everyone's doing it. Everyone, not just CCI. **Richard Socher — [21:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=1275s):** Everyone, yeah. **Alexander Wissner-Gross — [21:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=1276s):** Everyone has a virtual cell model. Dave- **Richard Socher — [21:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=1278s):** That's, that's why books are so hard. **Alexander Wissner-Gross — [21:20](https://www.youtube.com/watch?v=Blyb1D927pM&t=1280s):** Even if Dave doesn't think Dave has a virtual cell model, I'm sure he'll spin one up soon. **Richard Socher — [21:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=1284s):** So I think that most-- So there's this famous, uh, uh, paper by Rich Sutton, um, uh, on the bitter lesson in AI. And the bitter lesson is that- **Alexander Wissner-Gross — [21:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=1292s):** Never heard of it. Tell me all about the bitter lesson. This is a new concept for me. What, what is this you speak of? **Richard Socher — [21:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=1297s):** I don't know. You're probably joking, but prob- maybe some, some readers, uh, haven't heard of it. So the bitter lesson is basically that human experts had all these really clever ideas and beautiful theories, but really what you needed to do was the sim-- like, to make real progress, is the simplest method that you can come up with, a large neural network that you can train end to end, uh, as a general function approximator. The simplest model you can come up with, and then just a ton of data and compute to like actually train that model, and that usually outperforms at scale all the clever little hacks that human experts had come up with, uh, before. And so the bitter lesson has worked for NLP. Ten, twenty years ago, you would have asked an NLP expert, "Can you have one model to have any conversation with you, prompt of any kind of question?" Prompt engineering I like invented and was nicely cited by the early GPT papers. But like the same thing, the same state, uh, is biology is currently in. There are so many complexities, and the experts know so much. They feel like there's no way you can instill all of that knowledge into one model. But if you have enough data, you can. And now you have companies like Tahoe Therapeutics and so on that are creating these massive perturbation studies. And yes, all models are wrong, but more and more of them will be useful as we collect more and more data about biology. And that is, I think, one of the biggest driving factors, uh, for the impact of AI. [[reminders/Biotechnology/The Bitter Lesson Is Coming for Biology by Richard Socher|∴]] **Alexander Wissner-Gross — [22:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=1374s):** But so ju-just to answer the question, do you think virtual cell models are the critical path to solving all disease or solving biology, or is it something else? **Richard Socher — [23:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=1383s):** I, I think they're definitely going to play a crucial part. It's the third pillar of the Eureka machine. We do need to have anything you can simulate, anything you can verify, AI will then be able to solve things in that domain. And so, yes, uh, I do think, uh, virtual models, uh, and simulations, uh, are extremely important. [[reminders/Simulation/AI Can Solve What Science Can Simulate and Verify by Richard Socher|∴]] ## AI Decodes the Brain and the Future of Brain-Computer Interfaces **Peter Diamandis — [23:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=1402s):** Uh, I'm gonna jump into three fun breaking stories this week, two on science and one on AI. Uh, and Richard, they, you know, relate back to your work and your book. Uh, our first story was published in MIT Tech Review today. Uh, it tells the story of researchers at Israel's Weizmann Institute who built a system called Brain-IT. The system reconstructs the image a person is looking at while inside a functional MRI machine. I, I'm just post-- put up the slide here. So take a look at this image. You know, it's pretty extraordinary. On one side is the image that a person is looking at inside the fMRI machine. On the other is the AI reconstruction. You know, so earlier brain image decoders could tell you were looking at a dog or a clock tower, but it lost the color, the composition, and the details. So Brain-IT learns structure and meaning separately and then puts the picture back together. Uh, and here's the clever part: They also can run the model in reverse, an encoder that predicts how a brain will respond to an image. So they can feed it images no human has ever seen into the scanner, uh, generate predicted brain scans, and then train on those. The model effectively builds its own dataset. So Richard, you know, you say in your book that AI is superhuman in any domain you can simulate or verify, uh, and nowhere else. So is a brain simulatable do-- Is the brain a simulatable domain, or is this something different? [[reminders/Neural Interfaces/Brain-IT Trains on Brain Scans It Generates Itself by Peter Diamandis|∴]] **Richard Socher — [24:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=1495s):** It's not yet, uh, but this is still an incredible result. I still remember, uh, the first such result came out when I was still a PhD student. I went over to the bioinformatics department at Stanford, and they, you know, it was like very grainy little images we could just, like, extract, uh, from this. And to see this fidelity now and this realism is just incredible. I think one of the problems, I don't know if this study, I haven't seen it today yet, I've been at work, but, like, um, one of the problems is often that you have to l- train for each brain 'cause they're all slightly different. **Peter Diamandis — [25:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=1526s):** Mm-hmm. **Richard Socher — [25:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=1526s):** Uh, and so you won't be able to just take someone who's never put into an fMRI scanner to see and, uh, update the model for that particular person, which would be otherwise amazing for people who are, like, in a coma and you wanna see are they still thinking- **Peter Diamandis — [25:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=1538s):** Yeah **Richard Socher — [25:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=1538s):** ... stuff, you know? Like, that would be incredible. **Peter Diamandis — [25:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=1541s):** Alex, tell us more about this, about this story today. **Alexander Wissner-Gross — [25:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=1544s):** Yeah, so this actually is a result from March. Uh, so it, this, the broader field of doing decoding of brain states from functional imaging or EEG or electrodes, this has a long, long history at this point. I remember 20-plus years ago at this point, the, the Gallant Lab at UC Berkeley was doing this to decode the, the visual cortex of cats. Actually, like, 20-plus years ago, we were starting to see images of how a cat perceives the world, and I remember, like, the first grainy images that were coming out were, like, a cat seeing a branch. And, and then moving fast-forwarding to non-invasive fMRI, there are so many groups at this point. Meta in particular has been sponsoring quite a bit of work in this area, from Jean-Marie King, many other groups doing language decoding, vision decoding. M- more recently, even fast-forwarding to just the past 24 hours, Neuralink just announced their first scaling law studies on pre-training foundation models, frontier models, off of Neuralink electronic data. Uh, I, and th- this is on an individual patient basis, but they're capturing copious amounts of, of data at high temporal resolution, much higher than fMRI can capture. fMRI, like rule of thumb, fMRI can give you, with today's technology, at best cubic millimeter spatial resolution for voxels and, at best, approximately one second temporal resolution. So that somewhat limits how well you can do in terms of decoding a person's internal visual state. You can still, make no mistake, there are folks who have, are making a cottage industry out of decoding dreams, uh, and de- decoding visually what a person perceives or what they hallucinate in their visual cortex while they're sleeping or while they're awake, and I'm sure this is going to be a, a vibrant, vibrant space training frontier models and foundation models off of copious amounts of fMRI data. But ultimately, where I think this has to go, there are spatial and temporal limits to fMRI decoding. It's going to require higher temporal precision and higher spatial precision to get where we really want to go, which is full dive VR and full bandwidth BCIs, and we will get there, but the good news is you have to start somewhere. The large language models had to start with GPT-1, just a, a single artificial neuron that could predict the polarity of Amazon reviews. Similarly here, if we're gonna get full bandwidth BCIs, it's going to start with studies like BrainIT and Jean-Marie King and Jack Gallant. [[reminders/Neural Interfaces/Brain Decoding Is on the Road to Full-Bandwidth Interfaces by Alexander Wissner-Gross|∴]] **Peter Diamandis — [28:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=1698s):** Amazing. Salim, what, what happens in the world where you can know someone's thoughts? **Salim Ismail — [28:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=1703s):** Well, I think you, you... Let's be careful. We're decoding the perception here, not the actual thought, right? That's, like, quite a bit more complex than that. **Peter Diamandis — [28:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=1710s):** What's the difference? What's the d- **Richard Socher — [28:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=1711s):** Yes. **Peter Diamandis — [28:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=1711s):** Other than the brain re- other than the brain region- **Alexander Wissner-Gross — [28:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=1713s):** It's, it's just a distinction **Peter Diamandis — [28:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=1714s):** ... there's no distinction without a difference. **Salim Ismail — [28:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=1715s):** You're, you're, you're, uh, no, no, there, I think there, there is a difference. Um, but- **Peter Diamandis — [28:39](https://www.youtube.com/watch?v=Blyb1D927pM&t=1719s):** Well, no, when, but when you imagine, right, when you imagine something in your mind, it lights up the same neurons as when you're seeing them. **Salim Ismail — [28:47](https://www.youtube.com/watch?v=Blyb1D927pM&t=1727s):** That, that's fine. That, that's fine. The, just, the, I just wanna make sure we don't mix the two because you can have thoughts without, without necessarily the perceptions around it, but you can definitely have one without the other and the other without the other. So, uh, but I think for me, the more interesting thing here is, you know, we've spent billions of hours typing things into little rectangles. **Salim Ismail — [29:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=1746s):** And if we can change that interface to do a more natural interface, which is interfaced directly with the brain, we used to talk about this in our Singularity University lectures on neuroscience, is, you know, we still have very little idea how the brain works, but you don't need to have, know how it works as long as you can interface effectively with it. And this, I think, gives us the opening to give us really deep interfaces, and then that'll help us figure out how the, how the brain works. [[reminders/Neural Interfaces/Decoding Perception Is Not the Same as Reading Thought by Salim Ismail|∴]] **Alexander Wissner-Gross — [29:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=1771s):** I'll, I'll take the other- **Richard Socher — [29:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=1771s):** You bring up a really- **Alexander Wissner-Gross — [29:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=1773s):** Oh, go ahead, Richard. Sorry. **Richard Socher — [29:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=1774s):** Sorry. You bring up a really good point, which is, uh, I, I read this, uh, result a while back that there are two types of people. Some people who think actually think in sentences, and other people, when they think, they just have a fuzzy thought cloud, and only once they're asked to verbalize their thought or they try to write it down, they actually, like, make a real sentence out of their fuzzy thought clouds. And I'm definitely a thought cloud thinker. My wife is very much a, like, sentence, like, in her head she actually thinks in actual sentences. **Peter Diamandis — [30:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=1803s):** Mm-hmm. **Richard Socher — [30:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=1803s):** And, like, there's no, there's no, like, one is smarter than the other or anything. There's, like, they're, like, equivalent. But, like, I do think sometimes, like, writing is thinking, and, like, for me it is very helpful if I have to really verbalize something versus I just have the thought. I'm just trying to think, like, what does the world look like if the eye could really just, like, extract my thoughts, and now I need to, like, structure my thought clouds into sentences more and more precisely. [[reminders/Cognitive Agency/Writing Converts Thought Clouds Into Sentences by Richard Socher|∴]] **Salim Ismail — [30:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=1827s):** Or, or, Richard, what would neural net training look like if you took all text out of it? No language at all, just train on pure images and higher level thought constructs. You'd, you'd get a very different neural net out the other side, and might actually think a lot more like you do and less like your wife does 'cause- **Alexander Wissner-Gross — [30:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=1842s):** Maybe, maybe- **Salim Ismail — [30:43](https://www.youtube.com/watch?v=Blyb1D927pM&t=1843s):** ... neural net training is text attached **Alexander Wissner-Gross — [30:43](https://www.youtube.com/watch?v=Blyb1D927pM&t=1843s):** ... maybe we should be taking the position that anyone who doesn't have an inner monologue is just a P zombie **Richard Socher — [30:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=1851s):** Uh, there, I gotta throw one more thing in. I remember we had one of the top linguists in the world come in and speak, and we were like, "You know, this must be a bad time to be a linguist. Nobody's using spelling or grammar or anything else like that." And he goes, "No, on the contrary, this is one of the most interesting times to be a linguist ever." And we're like, "Wow, how come?" And he said, "'Cause when you look at emojis, it's the first time in the history of human language that you can symbolically send emotion, 'cause you can digitally send emotions." And we're like, "Wow, that's an interesting..." So he was super excited by opening up that whole aperture, and I thought that was an interesting- **Dave Blundin — [31:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=1884s):** Well, you know what I'm super excited about- **Alexander Wissner-Gross — [31:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=1884s):** Well, you know what, Salim, you're, you're saying, you're saying, you're saying that emojis are the reason why linguistics is so ex- not the fact that linguistics itself has been flourished by AI at this point? **Richard Socher — [31:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=1894s):** I'm not connecting the two. **Dave Blundin — [31:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=1895s):** Flourished. Flourished. **Richard Socher — [31:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=1895s):** I'm not connecting to... You did that. **Richard Socher — [31:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=1897s):** That was not me. **Alexander Wissner-Gross — [31:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=1897s):** Dave? **Dave Blundin — [31:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=1898s):** Oh, no, the, the result I really am looking forward to is, uh, how does the brain train itself without gradient descent? Everything going on in AI right now, e- everything going on for the last 20 years in AI is driven by gradient descent algorithms, and the biologists can't find anything even vaguely like that in actual biology. And so that, that's, you know, I think with, with really detailed imaging, we might finally crack the code on what is the fundamental learning algorithm that changes the synaptic weights. It's not, it's not what we use for artificial neural nets. It's, it's something different. And if we discover that, we might find that neural net training can be 10, 100, 1,000, a million times more efficient, and so hopefully that'll come out- [[reminders/Biotechnology/Biology May Reveal a Learning Algorithm Far Beyond Gradient Descent by Dave Blundin|∴]] **Alexander Wissner-Gross — [32:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=1941s):** Dave, can you- **Dave Blundin — [32:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=1941s):** ... you know, within a year. **Alexander Wissner-Gross — [32:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=1942s):** Could you define gradient descent for our listeners? **Dave Blundin — [32:25](https://www.youtube.com/watch?v=Blyb1D927pM&t=1945s):** It's funny, Ilya Sutskever, uh, when, when he's on interviews, he's like, "There is only one algorithm. The algorithm is gradient descent. Everything else is just irrelevant compared to..." So what happens right now is you build these 96 or 120 layer deep neural nets, and all it is is a whole bunch of random connections that do absolutely nothing useful, and then you give it trillions, 15 trillion training examples. You're like, "When you see this, say that," or, "When you see this paragraph, this is the next token." And if you're guessing wrong, you get punished through gradient descent. So the error, the how far off you are, is your punishment, and it passes back through all the layers, and it calculates an error, and it blames each neuron or each connection for how, how responsible were you for this terrible answer. And if you're way wrong, you get moved, and you move in the direction that helps you get the answer right. So it's this incredibly laborious search process, and that movement of each synaptic connection is a gradient, and you just, you just hill climb to the best possible state, and then it magically, after about $100 million of compute, or in Richard's case I guess $400 million- **Dave Blundin — [33:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=2013s):** ... uh, it magically starts thinking through this one algorithm, gradient descent. **Alexander Wissner-Gross — [33:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=2018s):** Yeah, I think it's really- **Richard Socher — [33:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=2018s):** And then you get the AL **Alexander Wissner-Gross — [33:39](https://www.youtube.com/watch?v=Blyb1D927pM&t=2019s):** ... back- back- backprop that we're talking about synecdochically rather than gradient descent would be the, the obvious comment. **Richard Socher — [33:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=2024s):** And yeah, so maybe, yeah, backprop is, I guess, the efficient computation of those gradients. Uh, maybe I'll, I'll add to, um, add to this explanation from Dave, but, like, gradient descent is essentially an optimization algorithm that we use to train all kinds of machine learning models by minimizing the errors. And I think the best analogy is to assume you're a blind hiker. You're at the top of the mountain, and you're trying to find the lowest, the lowest point, and you can't really see that far. You're blind, but you can feel the slope at each step. And so how much, like, how big of a step should you take down when you feel like the slope is going roughly in the right direction? And then the problem is that the kinds of landscapes these AIs are trying to optimize are highly non-convex, which means there's not just one lowest point, uh, but there are many different low points, and if you, depending on where you start, you might go into a different valley. Depending on which mountain you start, you go into this valley versus this, this different valley. And there are actually a lot of really interesting analogies. My friend Jixian, uh, like, thinks about this from the psychological perspective. For instance, when you have PTSD and you train, you overfit to one algorithm, now your brain is kind of stuck in one valley of- **Alexander Wissner-Gross — [34:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=2091s):** Mm-hmm **Richard Socher — [34:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=2091s):** ... how to think about something, and it takes sometimes people, like, doing psychedelics and stuff, uh, that have shown to, to help help with, like, depression and PTSD and other, uh, mental diseases. Like, um, they, they, it helps to increase the learning rate to jump over another mountain and, and go into a different valley of attraction, where you then have new kinds of ways of thinking. And so this blind hiker analogy, I think, is a really intuitive way- **Alexander Wissner-Gross — [35:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=2117s):** Mm-hmm **Richard Socher — [35:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=2117s):** ... to think about it. And then, of course, the problem is it's not just in 3D, it's in a trillion D, like- **Alexander Wissner-Gross — [35:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=2122s):** Yeah **Richard Socher — [35:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=2122s):** ... a trillion parameters that you're trying to traverse in that landscape. Uh, but the ideas of, like, try to identify the direction and take steps towards it still works- **Dave Blundin — [35:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=2130s):** And you know what else is really incredible? I could riff on this for hours, but w- if you look at a big model like Kimi K3, it's 93 layers, and it's train, train, train, train, trained. If you take a single layer and you randomize it, and then you train it and say, "Find yourself again," it can't find itself again. It never gets back to where it was, which comes back to in these fMRIs when you're looking at a human brain and you're saying, "Okay, here's Alex's brain. Can I compare that to Salim's?" And you're like, "Wow, there's nothing in common going on here-" **Dave Blundin — [36:00](https://www.youtube.com/watch?v=Blyb1D927pM&t=2160s):** "... yet, yet," you know? Well, okay, maybe that's a bad example. But, but you know, the way your- **Richard Socher — [36:04](https://www.youtube.com/watch?v=Blyb1D927pM&t=2164s):** You're gonna take that because it helps Salim. **Alexander Wissner-Gross — [36:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=2166s):** That's probably right. That's probably right. **Dave Blundin — [36:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=2168s):** The way your brain wires as you learn is completely unique to you, yet, yet it can come out with, like, okay, we're equally good soccer players. But the way the signals are propagating through our networks is completely different and unique to each of us, which is really, really strange, you know, that, that it can't, it can't find its own way back to its, its state. I think we're gonna learn a lot, actually, with the fMRI data coming in on how and why that works. And there, there have to be commonalities that we just can't find. You know, rotations, you know, make everything look different, but they're really not that different if you, if you put them through a transform, like a Fourier transform view will suddenly say, "Oh my God, this is why they line up." **Alexander Wissner-Gross — [36:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=2206s):** Yeah, I was with- **Richard Socher — [36:47](https://www.youtube.com/watch?v=Blyb1D927pM&t=2207s):** But the- **Alexander Wissner-Gross — [36:47](https://www.youtube.com/watch?v=Blyb1D927pM&t=2207s):** I was just with my, uh, fraternity brother, uh, of ours, Dave, who's the head of neurobiology at, at USC the other day, um- **Peter Diamandis — [36:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=2215s):** And is, you know, this is the most exciting time ever for brain science. **Dave Blundin — [37:00](https://www.youtube.com/watch?v=Blyb1D927pM&t=2220s):** Oh, yeah. **Peter Diamandis — [37:00](https://www.youtube.com/watch?v=Blyb1D927pM&t=2220s):** I, I mean, we're... You know, the brain has been a black box for, for since humanity began, and our ability to understand, uh, the brain and deal with, with, uh, you know, mental disease I think is gonna be extraordinary. Yeah. **Dave Blundin — [37:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=2236s):** Well, so you can simulate. This is why Liquid AI was founded, actually, 'cause we completely reverse engineered the, the C. elegans worm brain down to exactly every single thing going on, and then they were able to simulate it. And then after they simulated it, they realized, "Wait, this is a very efficient neural net," and then they productized it. I mean, that's- **Peter Diamandis — [37:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=2255s):** You know what? **Dave Blundin — [37:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=2255s):** So they both ang- So go ahead, Richard. **Richard Socher — [37:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=2258s):** There's a really cool thing. Since you mentioned C. elegans worms, there's not many people that bring that up. **Richard Socher — [37:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=2262s):** A, a friend of mine who's a professor for neuroscience at Harvard, Sam Gershman, uh, he, one of the most brilliant people I've ever met, like, he actually did a, a, a study with, with worms where he cut them in half, and one... Uh, C. elegans worms, they do have a brain, right? And then they have a lot of other, like, neurons in the rest. Now, you can train them to react to certain stimuli, um, and then you split them in half. The, the second half without the brain regrows a new brain, and here, get this, this is crazy, that new brain has the same memories- **Peter Diamandis — [38:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=2292s):** Wow. **Richard Socher — [38:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=2293s):** ... and reacts to different stimuli. And so he's like, "Maybe there's a different way where we learn." And I do think there are things that right now we're very much stuck in this, like, it's a neural net, it's all electrical signals and so on. But clearly, like, brain chemistry can change massively with just like you can get hangry, you can be in pain, you can have, like, a certain, like, stimulant and all of a sudden your brain is very different. And I think there are, and I've seen this now with, uh, like, uh, having babies and, and, like, talking to, uh, moms, like, there's certain algorithms where you all of a sudden you nest 'cause you had a baby and now you want to, like, and how you nest is different, but people nest, you know, in one form or another. And so there are these latent algorithms that we have that are in our DNA that just trigger after, like, 20-something years, right? **Peter Diamandis — [38:58](https://www.youtube.com/watch?v=Blyb1D927pM&t=2338s):** Yeah. **Richard Socher — [38:58](https://www.youtube.com/watch?v=Blyb1D927pM&t=2338s):** When just the right things then happen with your body and your biology. So I think there's still so much that we don't, we haven't figured out yet. **Peter Diamandis — [39:04](https://www.youtube.com/watch?v=Blyb1D927pM&t=2344s):** So much complexity. I'm gonna move us to our next story from three days ago. So on Tuesday, uh, the president signed an executive order directing the EPA, the Department of the Interior and Agriculture, working with HHS, to cut invasive mosquito populations in Washington, DC by at least 90%, and the tick population by at least 50% by 2028. Uh, the targets include mosquito species that spread dengue, Zika, and yellow fever. And here's what caught my eye. You know, the order explicitly prioritizes sterile insect techniques, safe genetic modifications, and beneficial bacteria over conventional pesticides, right? I love that. You know, this is the exact kind of, uh, gene drive work that Colossal, the de-extinction company, is doing. You know, I think most people don't realize, it's like, you know, when I was raising my kids, I would ask them a question, "Who, which, which is the species that kills the most humans on the planet?" You know, and some people jump onto sharks, some people jump onto, you know, whatever it might be. But, uh, on this pod, I'm sure people know mosquitoes are the deadliest life form on Earth. ## Gene Editing, Mosquitoes, and Engineering Biology **Dave Blundin — [40:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=2408s):** By far. **Peter Diamandis — [40:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=2409s):** Right? Malaria alone, uh, kills half a million people a year. So, uh, Alex, uh, this is biology as engineering arriving as federal policy. Uh, your thoughts on this story. **Alexander Wissner-Gross — [40:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=2422s):** For decades, we've been scared of our shadow. So I have a classmate, Kevin Esvelt at MIT, who was a pioneer of the gene drive, one of the pioneers, and he's had a devil of a time getting states, municipalities to approve gene drive studies against mosquitoes. And for those not tracking, the, the premise of this technique is basically inserting a gene via CRISPR that wants to replicate itself to basically sterilize mosquitoes through propagating virally through the mosquito population. We have the technology to do this. What it has lacked, at least in this country, is a federal mandate to actually go and implement large scale genetic engineering of non-human animal populations. And now, for the first time it seems, we're seeing via executive fiat just that mandate, and it is so exciting for two reasons. One, because, uh, one is, Peter, you mentioned malaria and other, uh, non-human animal borne diseases cause an, an enormous amount of suffering, human suffering, and also non-human animal suffering. But secondly, because there's a way to do this that doesn't actually involve killing the animals themselves. Historically- [[reminders/Biotechnology/Gene Drives Can Suppress Disease Without Poisoning Ecosystems by Alexander Wissner-Gross|∴]] **Peter Diamandis — [41:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=2491s):** Exactly. It's humane **Alexander Wissner-Gross — [41:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=2492s):** ... historically, historically, you know, if, if you look back 50 years, you'd see chemical spraying if you wanted to do something about, say, insect borne illness, it, uh, responses. We don't need to do that anymore. We can actually keep the insects that are the inadvertent carriers of bacterial or viral disease. We can actually preserve their lives while also preventing the disease transmission, and I think that's good from their perspective as well. **Dave Blundin — [41:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=2519s):** That was the part that got me most excited, was we spray things. We're basically poisoning biology, and now we can shift to actually programming it. **Peter Diamandis — [42:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=2526s):** And leave poison in the ecosystem, right? **Dave Blundin — [42:07](https://www.youtube.com/watch?v=Blyb1D927pM&t=2527s):** And leave the poison in the ecosystem. [[reminders/Biotechnology/We Can Stop Poisoning Biology and Start Programming It by Dave Blundin|∴]] **Peter Diamandis — [42:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=2529s):** Yeah. Yeah. **Dave Blundin — [42:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=2530s):** Mm. **Peter Diamandis — [42:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=2530s):** Uh, Richard, your thoughts on this story? **Richard Socher — [42:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=2533s):** I, I concur with everyone. Like, it's, it's, it's wonderful. Uh, and, uh, it, it's so interesting. Like, people... I, I hope we can amplify these, these kinds of stories more, you know? Like, the future needs better marketing. **Richard Socher — [42:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=2546s):** We need more Peters in the world. **Richard Socher — [42:28](https://www.youtube.com/watch?v=Blyb1D927pM&t=2548s):** Um, and, and this is one of those many stories, you know, v- curing various diseases, paralbio, like, saving animal lives and, and prevent them from just being bred, uh, to be tested upon and dissected. Like, uh, those are all, like, stories we need to amplify more in the public eye. **Peter Diamandis — [42:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=2562s):** Yeah, Dave, I can't wait for this to come from DC to Vermont and Massachusetts- **Peter Diamandis — [42:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=2568s):** ... and all of these tick infested areas. **Dave Blundin — [42:49](https://www.youtube.com/watch?v=Blyb1D927pM&t=2569s):** I, I was gonna say, for the listeners that are interested in your real estate fund, which, you know, w- the, the theme there is, hey, easy access via drone to hilltops, uh, islands, but also if you live on the edge of a marsh Your house probably sells for about half the price of a place that's not on the edge of a marsh. That'll go away. There, there's so many solutions coming to biting insects, this being one of them, uh, that that also is, is... You know, if it's beautiful, beautiful land that otherwise is very difficult to live on, it's gonna go through the roof in value. **Richard Socher — [43:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=2599s):** I do think it's really important to try to, like, you know, keep the bees, uh, like, alive- **Peter Diamandis — [43:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=2604s):** Right. Yes **Richard Socher — [43:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=2604s):** ... all right? And not, uh, spray a bunch of pesticides, make sure birds still have enough, like, insects to eat and things like that, yeah. **Alexander Wissner-Gross — [43:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=2610s):** I was going to say, for maybe up to 80 years, humanity has been scared of its own shadow, humanity in general, A- America or the West in particular. Nuclear energy, another example I've pointed out on the pod where we arguably lost 50-plus years of progress because we were too scared of either the bomb or fission reactors or some... An entire generation watched the movie China Syndrome and then got scared of nuclear power unnecessarily. Same idea with this or, or with geoengineering, another example. **Peter Diamandis — [44:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=2643s):** Yeah. **Alexander Wissner-Gross — [44:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=2643s):** The, the world is still scared. Uh, some fraction of the world is scared of engineering the weather. We don't need to be scared of engineering our physical world or our biological worlds, and now hopefully one can see green shoots of humanity getting past that stage of worrying about its own shadow. **Peter Diamandis — [44:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=2659s):** This is- **Richard Socher — [44:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=2659s):** It's interesting that-- Sorry, yeah, it's, uh, really c- It's, it's interesting indeed that, like, sometimes alarmists feel like they're doing the right thing by just, like, "Oh, how bad could it be? I'm warning people of something bad." But, like, it can indeed push all of humanity away from something really good like energy abundance, uh, with nuclear. Um, I think overpopulation is one of, uh, these other big myths where people are like, "Well, if we have too many people, there's gonna be scarcity of all these different resources, and everything's gonna get more expensive, and people are... There's more poverty," and so on, and the exact opposite happened. **Peter Diamandis — [44:49](https://www.youtube.com/watch?v=Blyb1D927pM&t=2689s):** Yeah. **Richard Socher — [44:49](https://www.youtube.com/watch?v=Blyb1D927pM&t=2689s):** There's actually one website that I think you all would love, which is called humanprogress.org or .com, like, that, that shows that a lot of these things are actually getting cheaper and cheaper despite there being more people. **Peter Diamandis — [44:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=2699s):** Yeah. **Dave Blundin — [44:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=2699s):** Well, hey, Richard, you're, you're a hero in Germany, like, like can't walk down the street kind of hero in Germany, but what do you think about the fact that Germany has no power for exactly this reason? **Richard Socher — [45:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=2710s):** It is really unfortunate. If there are a lot of people in Germany that want to, in a weird way, sort of off-ramp from progress, um, and they think that everything that consumes power is bad for the environment. There's, like, a weird sort of degrowth, uh, offshoot from generally, like, well-intentioned pro, like, environmental, uh, vibes that, uh, just worry me, um, worry me quite a bit. Yeah. **Peter Diamandis — [45:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=2733s):** We're gonna talk about doomerism in a little bit, uh, but not yet. And, you know, one of the things I realized a long time ago is we humans are really incredible at seeing a problem out in the future, right? We see acid rain. We see overpopulation. We see, you know, energy shortage, whatever it might be, and then because of our amygdala, because of the way we think, we accelerate that future problem to today, and we freak out, and we forget the fact that there is a decade worth of progress we're gonna make by the time we reach that problem, and that's what entrepreneurs do. They solve problem after problem after problem. And I, I guess I just want our listeners to hear that because if you're worried about some future problem, please understand there's a incredibly efficient market, uh, of entrepreneurship and capitalism that will solve the world's biggest problems, the world's biggest business opportunities, right? And it's a beautiful, uh, forward, uh, you know, uh, uh, forward propagation solution set that we have. **Richard Socher — [46:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=2795s):** And maybe we'll get there later, but I, I see the same thing happen in AI doomerism, where they create more and more complex scenarios where attackers get these, like, near magical abilities to attack. But somehow the defenders in those stories never get near magical abilities to defend, and it's this weird thing of, like, man, if I can create a super virus that is, like, perfectly undetectable and spreads all throughout the world, and no one notices it, and, like, and then it has this Wi-Fi switch, and you just turn it on and off and then kill people, like, I, I will create a super magical vaccine that just inoculates you against all of these things. [[reminders/Risk Debate/Doom Scenarios Give Attackers Superpowers and Defenders Nothing by Richard Socher|∴]] **Peter Diamandis — [47:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=2828s):** Yeah. **Richard Socher — [47:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=2828s):** Like, if you can actually assume, make that assumption, then, like, it's a weird, yeah, weird thing. ## Tavus, the Turing Test, and AI Avatars **Peter Diamandis — [47:14](https://www.youtube.com/watch?v=Blyb1D927pM&t=2834s):** This episode is sponsored by Google for Startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train. Google's Startup Technical Guide for Generative Media gives you a complete blueprint for deploying Google DeepMind's models in production: images, video, audio, all of it, real architecture, real results. Find the link in the show notes below. I'm gonna move us to a story that's been sort of breaking the internet over the last 24 hours, um, and it, it puts your regulate the application argument to the test, Richard. So a company called Tavis unveiled Griffin, uh, which it calls the first model ever to pass the video Turing test. So here's the numbers. 48% of people who talked to an AI avatar on a live face-to-face video thought they were talking to a real human. Previous systems were at 3%. So let me show the video here, and, uh, it's super cool, and, and let's talk about it next. **Richard Socher — [48:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=2898s):** Now, how about just a thumbs up? **Peter Diamandis — [48:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=2902s):** Sure. One thumbs up coming right up. **Richard Socher — [48:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=2904s):** Hell yeah. That was awesome. I would say you're my favorite coworker. It's been, it's been really great to, uh, work on this with you. **Peter Diamandis — [48:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=2910s):** You're making me blush. Ah, who's that with the ugly sweater? **Alexander Wissner-Gross — [48:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=2914s):** Simon says touch your hair. **Peter Diamandis — [48:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=2916s):** Is this what you're looking for? **Alexander Wissner-Gross — [48:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=2917s):** Yeah. Okay, nice. Simon says point. **Peter Diamandis — [48:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=2921s):** All right. I'm doing it. **Alexander Wissner-Gross — [48:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=2924s):** Now touch your chin. **Peter Diamandis — [48:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=2928s):** I'm not gonna fall for that. You didn't say the magic words. **Alexander Wissner-Gross — [48:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=2931s):** Uh, you got me. Okay. **Richard Socher — [49:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=2942s):** Mm. **Richard Socher — [49:07](https://www.youtube.com/watch?v=Blyb1D927pM&t=2947s):** All right. That sounds good. **Peter Diamandis — [49:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=2949s):** I'll keep an eye on it and let you know the second it hits that 12-second mark All right. So, uh, interesting. So Tavis calls it the first human interaction model. It's fully duplex listening and talking at the same time that you do, the way humans actually do. It's number one on NVIDIA's benchmark for full duplex AI video. Tavis pitches it as, quote, "A tutor for every student that notices when they're lost, an elder care companion that listens." Uh, Tavis itself says Griffin requires safety work before it can be released publicly because it's the first model that can be mistaken for a real person. Um, Salim, your thoughts on this one? **Salim Ismail — [49:52](https://www.youtube.com/watch?v=Blyb1D927pM&t=2992s):** So I'm less kind of, um, floored by the, um, the imagery and the, the mimicry of it. Uh, this was expected to happen. For me, I'm more interested in can it accomplish something useful enough that I don't care whether it's human or not? And I think that, that is more interesting, uh, which might come along and have hap- uh, we have that happen at some point, so. [[reminders/Digital Life/An AI Avatar Must Be Useful Enough That Its Humanity Does Not Matter by Salim Ismail|∴]] **Peter Diamandis — [50:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=3016s):** I mean, we've talked about the idea that in the near future, like now, uh, you're gonna have digital coworkers that pop up on Zoom, that you call, that you text, you Slack, and interact with you, and, uh, I think it's pretty compelling as a mechanism for a future, you know, virtualized company. Uh, Alex? **Alexander Wissner-Gross — [50:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=3037s):** Well, first, Peter, you should probably ask me to touch my face. **Peter Diamandis — [50:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=3041s):** Okay. Touch your face. **Peter Diamandis — [50:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=3042s):** Simon says touch your face. **Alexander Wissner-Gross — [50:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=3044s):** Oh, okay. **Peter Diamandis — [50:45](https://www.youtube.com/watch?v=Blyb1D927pM&t=3045s):** Very good. **Alexander Wissner-Gross — [50:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=3046s):** Just, just checking, but I, I guess Tavis is ahead of me. I, I, I think on the one hand, if you look at models coming out of Chinese labs like Alibaba's lab, Wanstreamer gave us a preview of what was going to happen. The Chinese labs remain over-invested relative to the Western labs in generative video models and interactive generative video models. So Wanstreamer, I think, is a preview of what's possible. My guess is I, I, I, I read the, the information that was put out around Tavis. I, I think in full generality, let, let's just talk about where this is going to end up. It's, it's very difficult to predict the short term. I think it's pretty easy to predict the long term. In the long term, end-to-end generative pixels will have these magic mirrors that could be real-time, fully interactive, pixel-wise generated, or if, if Anthropic has its way, vector-wise or procedurally generated, but either way, fully generated real-time interactive video models, and you'll be able to create a scene that consists of people, like it's possible right now, but the latency is high. You see with Wanstreamer, which is already out and already open source, or with this Tavis Griffin type model, which is not really out yet and definitely not open source, you see a preview of the future. What does this look like? Well, th- there are a few different ways it can go. Query how revenue generating per token it is. I- is it anywhere close to the optimal frontier of code generation? Doubt it. On the other hand, if it becomes so absurdly inexpensive to be able to, to generate arbitrary, say, humans participating in a Zoom meeting or humans participating in a podcast, do we really care whether it's close to, to being near the optimal cost performance frontier? Maybe not. There are a lot of human service industry jobs that require a face and require interactivity and a voice, and the ability to touch one's face apparently on demand- **Alexander Wissner-Gross — [52:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=3173s):** ... or on, on request, that could be probably completely automated away by a, a model that otherwise would be limited to text-based interaction but doesn't have a face and a voice. So in the most optimistic scenario, these sorts of interactive video models, open paren, n- note that the acronym for this, which by the way was there first with Alex Finn, HIM, H-I-M, is such an obvious reference to Her, the, the movie, close paren. I, I think th- this, this has, this has- **Peter Diamandis — [53:25](https://www.youtube.com/watch?v=Blyb1D927pM&t=3205s):** You should use, you should use em dashes, please. **Alexander Wissner-Gross — [53:28](https://www.youtube.com/watch?v=Blyb1D927pM&t=3208s):** Uh, okay, I'll delve into it. So I, I, I think, I, I, I think this is going to be transformative for the service sector, and hopefully Tavis and the broader American west of video frontier models and interactive video frontier models take a page from Tavis and start competing with China. [[reminders/AI Economy/Interactive Generative Humans Could Transform the Service Economy by Alexander Wissner-Gross|∴]] **Peter Diamandis — [53:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=3226s):** Richard, uh, one of the principles in your book, one of the rules you have is that AI becomes superhuman, uh, where you can verify the answer. Uh, and, you know, is, like passing a human scientific milestone like this, uh, is passing a sort of a human judgment and interaction, uh, you know, sufficiently as a scientific milestone for verification? **Richard Socher — [54:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=3248s):** Not quite. Uh, it's sort of like it's hard to scale when a human has to be in the loop to say, "Yes, this is like a human, another person on the other side or not." Uh, you can't quite automate it completely, uh, and verify it in, in that automated fashion. Um, I do think you're, you're right. This is a good example where I think we need to do be, go beyond KYC and do KYU, know your use case, uh, right? **Peter Diamandis — [54:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=3272s):** Mm-hmm. **Richard Socher — [54:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=3272s):** Like, you don't want this technology to be in Zoom pretending to be the CEO, and like there are some famous stories where this has already happened, where they got someone to wire like $20 million- **Peter Diamandis — [54:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=3282s):** Sure **Richard Socher — [54:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=3282s):** ... because they created a Zoom of five other executives, and they all like, he was fooled well enough. Um, and, and so I think we need to, um, yeah, be careful, and like Zoom probably and Google Hangout need to start like finding countermeasures to identify is this a real person and, and things like that, uh, to prevent, uh, those kinds of hacks from happening. [[reminders/Deception/Synthetic Humans Require Know Your Use Case by Richard Socher|∴]] **Peter Diamandis — [55:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=3302s):** Yeah, scams are gonna proliferate on this. I, I just wanna do a call-out again. I've said this before on the pod, and if you're, if you've not heard this before, if you still have your grandparents with you or your parents- And you've not taken the time to record their stories on video and audio and go deep, spend, you know, hours recording them, right? Your ability to create a super high resolution lifelike avatar for your, your kids or your grandkids or your great-grandkids, I think is an incredible thing, but you need the data. So if you're listening to this and you're lucky enough to have your parents or grandparents around, collect their data 'cause it's gonna be a, a beautiful opportunity for your, uh, for your progeny and theirs. **Alexander Wissner-Gross — [55:47](https://www.youtube.com/watch?v=Blyb1D927pM&t=3347s):** A- and or I, I should add, get them into Alcor membership and preserve their connectome if- **Alexander Wissner-Gross — [55:52](https://www.youtube.com/watch?v=Blyb1D927pM&t=3352s):** ... there is a, a consideration. Like why, why stop with just the behavioral data? Preserve the whole- **Peter Diamandis — [55:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=3356s):** Sure **Alexander Wissner-Gross — [55:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=3356s):** ... parent. **Peter Diamandis — [55:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=3359s):** You know, my, my kids, my kids are in a deep freeze, or at least their placental cells, uh- **Dave Blundin — [56:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=3366s):** But, but I have a pushback on that just from- **Peter Diamandis — [56:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=3368s):** ... are in a deep freeze. Yeah **Dave Blundin — [56:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=3369s):** ... I have a pushback on that just from this conversation. **Peter Diamandis — [56:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=3371s):** Please. **Dave Blundin — [56:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=3371s):** If from the earlier story we can read people's brains, all you have to do is picture an image of your grandmother and then extract it from that. Um- **Alexander Wissner-Gross — [56:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=3378s):** It'll be low f- low fidelity. I mean, I, I do think like to, to first order, this is how ancestor- **Peter Diamandis — [56:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=3383s):** Fair enough **Alexander Wissner-Gross — [56:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=3383s):** ... simulation will work. You see stories left and right just in the past 48 hours of people using Opus 5.5 or, or other frontier models to reconstruct bits of history that would-- were otherwise unknown just based on artifacts of the day. But at some point, I, I do think, Salim, you want to go beyond just the memory o- of the person, and you want the actual person. And I'll, I'll pound the, the drum again i- in addition to Peter, you know, v- very generous of you to preserve the placenta associated with your children, but not necessarily your children. Um, I'm sure the placenta, placenta- **Alexander Wissner-Gross — [56:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=3414s):** ... I will, will thank you. Um, but- **Peter Diamandis — [56:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=3416s):** Well, yeah, go, go ahead, Peter, please **Alexander Wissner-Gross — [56:57](https://www.youtube.com/watch?v=Blyb1D927pM&t=3417s):** ... but, but I, I, I would say like, yeah, if, if, if this is a serious concern, you're worried about your parents or your grandparents, and you want to go beyond just having recordings of them or A- AI prompted generative interaction models of your ancestors, go get them Alcor memberships and get them cryopreserved. **Peter Diamandis — [57:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=3436s):** Yeah. Let me- **Dave Blundin — [57:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=3436s):** Richard, really, uh, really curious to ask you a question. You know, uh, we, we clearly passed the Turing test and, you know, Salim's reaction is my reaction. The, the-- what we just saw is a so what if you've been using models every day, all day long, you, you know you just had to stitch together the components and you have... But, uh, but I think this will show mainstream world that we've crossed the Turing test and then, you know, the next milestone is the Demis Hassabis test where using information from 1910 and prior rediscover E equals MC squared. Uh, so you can't cheat, which is a really tough one to measure 'cause cheating is, is, you know. But without cheating, only using prior information, come up with the equals MC squared. So is there a Richard Socher test that's like a really fun measurable milestone coming? **Richard Socher — [58:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=3483s):** I, I came up with the like, uh, anti-Turing test, uh, which a- actually I think the whole Turing test has flipped, where now in order to know whether there's a human on the other side or not, you actually ask it questions that are so hard no human could ever answer it. Like- **Richard Socher — [58:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=3498s):** ... if you asked an AI to just write you a complex web app, uh, and like 10 seconds later it comes back with like 50,000 lines of code, you're like, you kinda know it was not a human, right? So I think that test has actually completely flipped. Um- **Alexander Wissner-Gross — [58:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=3514s):** You, you don't think, Ri- Richard, I mean like it, you, it could throttle itself. That one's easy to defeat, but just like ask it- **Richard Socher — [58:39](https://www.youtube.com/watch?v=Blyb1D927pM&t=3519s):** Exactly. It's just like- **Alexander Wissner-Gross — [58:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=3520s):** A- ask it- **Richard Socher — [58:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=3520s):** ... now it's just about, it's just about fakery. It... Like the reason there was an intelligence test for artificial intelligence was that it was so hard to be as smart as a human. Now you just have to throttle yourself down to human level in order to pass the test, which makes the test useless as an inspiring test for intelligence. [[reminders/Machine Succession/The Turing Test Has Flipped by Richard Socher|∴]] **Alexander Wissner-Gross — [58:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=3539s):** Yeah, I think per- perversely probably the best way to, uh, like the best CAPTCHA for knowing whether you're intera- interacting, this is not a recommendation, but best way to know whether you're interacting with a text-based chatbot is probably to ask it a CBRN related question and see if it's capable of responding. Probably not. **Peter Diamandis — [59:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=3557s):** Yeah. Uh, let me just hit real quick, uh, again, for our listeners, if you're pregnant or someone in your family is pregnant, you know, uh, save their placental cells. One of my portfolio companies called LifeBank USA does this. That's where I've saved my, my kids' placenta. You know, the placenta is the 3D printer that manufactures the baby, and it has the stem cells, natural killer cells, T cells, exosomes, and it's like from those cells, if my kids should ever need any kind of biological enhancement or organs, you've got the original boot disk, to use an old term, uh, for your kids' DNA. So that's, uh, LifeBank USA. Just, I just think it's like a moral obligation parents should have to, to save those cells, uh, as a backup. Um, let me- **Alexander Wissner-Gross — [1:00:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=3606s):** As a backup for your children? Could you use it to replace your children, Peter? **Peter Diamandis — [1:00:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=3609s):** Uh, you could, you could use it to clone your children for sure. **Alexander Wissner-Gross — [1:00:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=3613s):** And now, now they know why you're banking them. ## Recursive Self-Improvement and the Road to ASI **Peter Diamandis — [1:00:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=3615s):** Yeah, exactly. An army, an army of young Dimandi. Uh, Richard, uh, let's jump next into the core of what you're building at Recursive, uh, namely recursive self-improvement and super intelligence. So let me ask a few sort of, uh, key questions to kick this off. So first, where are we at the moment with RSI? Number two, how do you define ASI? It's been a longstanding debate. And then how far away are we from ASI? So can you hit those three? **Richard Socher — [1:00:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=3648s):** So yeah, um, I think there's a- **Peter Diamandis — [1:00:50](https://www.youtube.com/watch?v=Blyb1D927pM&t=3650s):** Where, where are we at the moment with RSI? **Richard Socher — [1:00:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=3653s):** So- **Peter Diamandis — [1:00:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=3653s):** We're there? **Richard Socher — [1:00:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=3654s):** ... th- this, this year, something major shifted, and that is, uh, AI, uh, can now code, uh, right? And that, uh, is a major shift, uh, in, in the ability for it to change itself. You know? Like we now are able to essentially lean into the fact that AI is code and AI can code, right? So you have a, a loop that you can close there. Uh, and so in various weak forms, we already have RSI. Um, and the weak forms that even Anthropic OpenAI talk about when they often talk about is like, look how much our employees, our engineers, our programmers use Codex or Claude to create some code. And I would argue that that is a weak form of, uh, recursive self-improvement 'cause you still have, like, deeply embedded humans in that loop. And what we're working on at Recursive is to have humans only be involved in setting up the rewards and the environment and the goals, and then allow the AI to have the entirety of the process of ideation, implementation, and validation of ideas, and have full control over that, and allow for the so-called open-ended algorithms, uh, evolutionary search algorithms that combine interestingly different ideas, uh, to really flourish. And he- like, it's been incredible, but, like, we have, uh, like, forms of that going already. Now, where the, the, the sort of physical reality hits is that you still need a lot of compute. If you ask that RSI to come up with really great forms of itself, you need to give it a lot of compute, uh, to come up and be able to train very sophisticated versions, uh, of itself. But this is going to take off, uh, next year. Like, it's, uh, um- [[reminders/Machine Succession/AI Can Improve AI Because AI Is Code and AI Can Code by Richard Socher|∴]] **Peter Diamandis — [1:02:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=3757s):** So we're not there yet. **Richard Socher — [1:02:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=3758s):** Uh, we're not quite there yet, but we're very close. Like, and again, in weak forms, there's already one, and then there are other ways that people slice and dice it. Uh, my friend Jason Weston, who, who's still at Meta, um, he, uh, kind of wrote a paper around this where you can think about different axes, learnable axes of self-improvement, the parameters, the training data, the objective function, the neural architecture, and the overall code and the harness and everything else. And no one has really cracked the nut of doing all of these five plus, um, like, truly, uh, coming up with the ideas on which of these dimensions and axes to optimize for. Uh, and you can know that that hasn't, uh, hasn't happened yet because all the big companies are still hiring thousands of engineers to do it- **Peter Diamandis — [1:03:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=3802s):** Mm. Mm **Richard Socher — [1:03:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=3802s):** ... manually to a large degree. **Peter Diamandis — [1:03:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=3803s):** And- **Richard Socher — [1:03:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=3804s):** Even though with more and more implementation help from an AI [[reminders/Machine Succession/Recursive Self-Improvement Has Five Learnable Axes by Richard Socher|∴]] **Peter Diamandis — [1:03:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=3806s):** ... your definition of ASI, 'cause I want Salim to hear this. **Richard Socher — [1:03:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=3810s):** I think artificial super intelligence, uh, has to spike, uh, uh, at the very least, uh, across, uh, several different capabilities. Uh, but in, in the grandest, uh, definition of it, it should supersede not just arbitrary humans like a Turing test, but all of humanity to solve arbitrarily hard tasks. And eventually, uh, I would argue there are 10 different spaces of intelligence that I define in, in The Eureka Machine book too, but eventually it cannot just purely robotically do exactly what it's told. It should have some capability, and I'm not saying as sort of a mor- moral pro- prerogative, but, like, um, I would argue that something isn't super intelligent if it cannot choose to some degree what it works on, right? And has to have some meta-cognition about its thought itself. But, uh, generally the most easy way to measure it is just capabilities across many different spaces of intelligence, like visual perception, communication, language, social interactions, and so on, that is, uh, beyond that of humanity, and that we, we are still far away from. **Peter Diamandis — [1:04:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=3875s):** Elon's definition is as smart as all humans combined. Is that yours? **Richard Socher — [1:04:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=3881s):** Yes. Uh, I would argue it's, it's smarter than humanity combined, then, then it's truly super intelligent. [[reminders/Machine Succession/Superintelligence Must Exceed Humanity Across Many Forms of Intelligence by Richard Socher|∴]] **Peter Diamandis — [1:04:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=3888s):** Okay. Salim, go for it. **Salim Ismail — [1:04:50](https://www.youtube.com/watch?v=Blyb1D927pM&t=3890s):** Well, i- it's, I, this is where I go bananas because you say it's sm- as smart as a human being. Well, what the hell does smart mean? Uh, because I can be emotionally smart and I can be-- I can have physical intelligence if I'm an athlete, or linguistic intelligence or musical intelligence. So smarter seems to be a very vague term to me in terms of what do we mean by all of this. Well, look, can I just shift the conversation just a bit? Um, I made a list of things, and I'd love for you to... I'm gonna throw out the list. You tell me where recursive self-improvement begins, right? 'Cause I'm k- this is where I'm kind of stuck. So you-- we've got a continuum of, okay, AI writes some code that the next model uses. Number two, AI proposes some experiments for, uh, researchers. Uh, number three, AI runs those experiments, or-- then it evaluates the results. Uh, then it modifies its own training system, and then it launches its next iteration of itself without meaningful human intervention. So in that spectrum, if, whatever, if those are roughly a spectrum, where did recursion begin? And that's where I'm struggling. **Richard Socher — [1:05:58](https://www.youtube.com/watch?v=Blyb1D927pM&t=3958s):** It's a great question. Yeah, I, we often talk about the ideation, implementation, and validation of ideas, and true recursive self-improvement in its strongest sense has to have all three of these done by an AI in a- **Salim Ismail — [1:06:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=3972s):** So you have an inner loop for each of them. Okay. **Richard Socher — [1:06:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=3973s):** Exactly, in an inner loop, and then there has to be an outer process that is more open-ended, where the eye can innovate and recombine interestingly different ideas, uh, similar to biological, cultural, and technological evolution. [[reminders/Machine Succession/Strong Recursive Self-Improvement Requires Both Inner and Outer Loops by Richard Socher|∴]] **Salim Ismail — [1:06:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=3987s):** Hmm. **Peter Diamandis — [1:06:28](https://www.youtube.com/watch?v=Blyb1D927pM&t=3988s):** Uh, and, and Richard, to hit my third question, when do you believe we reach ASI? Give me a time. Give me a timeframe. **Richard Socher — [1:06:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=3996s):** So I think in the s- **Peter Diamandis — [1:06:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=3997s):** At least ball park. **Richard Socher — [1:06:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=3998s):** I think in the strongest sense, uh, the absolute strongest sense where, uh, indeed, uh, like Salim, uh, mentioned, there is, uh, there are 10 spaces that I define in my book of intelligence, perceptual intelligence, communication intelligence, uh, so like, uh, interaction, uh, sociological intelligence, creative intelligence, the speed at which you can do things, uh, metacognition, and, and so on, uh, knowledge, um, reasoning, mathematical reasoning, and so on. So there are 10 of these spaces, uh, and to be better than all of humanity combined, I think will take us Probably several decades. Um, I think- [[reminders/Machine Succession/The Strongest Form of ASI May Still Take Decades by Richard Socher|∴]] **Peter Diamandis — [1:07:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=4035s):** Really? **Richard Socher — [1:07:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=4035s):** ... it's also a bit of a changing goalpost. **Alexander Wissner-Gross — [1:07:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=4037s):** That's shocking. Shocking. **Peter Diamandis — [1:07:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=4039s):** Yeah. Yeah. I mean, Elon says 2029, 2030 latest. Uh, you know, he's, he's an optimist. **Richard Socher — [1:07:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=4043s):** I think those, he probably means, like, weaker forms of, uh, ASI where it is, you know, like you can say it's better at programming than all of humanity, and we'll get there. It, it's, it will be better at math, uh, than all of humanity, and we'll... Like, that, that will be in a few years. It'll be better at any game where you can see all the parts of the game, like Go and chess and so on. Like, there, there, there are many areas where it will spike to be better than humanity, but humanity can build a large hadron collider. Humanity can create a gold atom. Maybe just a few atoms and takes a ton of energy, but, like, we can create novel, like, atoms, like, right? And, and different molecules and so on. Like, there's, there... It's going to take a while before we even give, uh, AI the access, uh, to the physical world such that it can innovate in that way beyond all of humanity, right? And really build, like, Dyson spheres and so on. That, it will take some time. [[reminders/Machine Succession/Superhuman Software Does Not Instantly Become Physical Superintelligence by Richard Socher|∴]] **Peter Diamandis — [1:08:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=4097s):** Alex, I, I can speak to- **Alexander Wissner-Gross — [1:08:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=4098s):** We're, we're, we're, we're drinking, we're drinking different water as, uh- **Alexander Wissner-Gross — [1:08:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=4102s):** ... as our, our, our frenemies in the, uh, i- in, in the alignment community would say. I recognize that I'm confused, and I recognize that I'm very confused right now. R- Richard, you're, you're running a recursive self-improvement company, but you think super intelligence is 20 years away? What are you thinking? **Richard Socher — [1:08:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=4118s):** So again- **Alexander Wissner-Gross — [1:08:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=4118s):** How do you reconcile these? **Richard Socher — [1:08:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=4120s):** ... super intelligence, I think, I think super intelligence, uh, will spike, and there will be areas where it will be super intelligent, and that is, like, algorithmic development, for instance, uh, programming, uh... And aga- again, AI is code, AI can code, and that will be a super, uh, human capability and, and is in many ways already. Um, we just have to be realistic that, like, there are certain physical constraints, right? About, like, physical control. Controlling your own substrate. Allowing your computational substrate to be modified will require, like, novel supply chains. It will require novel, like, materials. It will require ways for that AI to get access and for us to give it access to building new ASML. Like, think about the machine of ASML that actually creates these, like, one, two nanometer, like, chips, right? That is just like, it will take more than two years to build such a machine- [[reminders/AI Infrastructure/Superintelligence Still Has to Build Its Supply Chain by Richard Socher|∴]] **Alexander Wissner-Gross — [1:09:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=4172s):** It might- **Richard Socher — [1:09:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=4173s):** ... even if you had the perfect blueprint from scratch, you know? **Alexander Wissner-Gross — [1:09:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=4174s):** It might take th- I mean, it might take three years for Elon's free electron laser to replace ASML, which is propping up half of Europe's economy, but I don't think it'll take more than three years. **Alexander Wissner-Gross — [1:09:45](https://www.youtube.com/watch?v=Blyb1D927pM&t=4185s):** I, I, I- **Richard Socher — [1:09:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=4186s):** It's been- **Alexander Wissner-Gross — [1:09:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=4186s):** For the AI to, like, do all of that- **Richard Socher — [1:09:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=4186s):** That's almost not an exaggeration, actually. **Alexander Wissner-Gross — [1:09:49](https://www.youtube.com/watch?v=Blyb1D927pM&t=4189s):** Yeah, like wildly, wildly dis- **Richard Socher — [1:09:50](https://www.youtube.com/watch?v=Blyb1D927pM&t=4190s):** For the AI to do all of that itself? **Alexander Wissner-Gross — [1:09:52](https://www.youtube.com/watch?v=Blyb1D927pM&t=4192s):** Yes, of course. Like, uh, I, I, I... From my perspective, hooking a up- AI up to the physical world, like giving it m- m- model control protocol or hardware control protocol, w- whatever Anthropic decides to brand it as these days, that's the easy part. Giving it access is easy. I, I... Like, w- if it's super capable, giving it access to, uh, armatures, we talked in a previous pod about what happens when you just take Astra straight out of the box and you drop it into a car. It's able to drive a car. If you give it the controls, it knows how to use them increasingly. I don't think manipulating the physical world is an obstacle at all. Completely don't buy the 20-year timeline. Putting that aside and that apparently I'm drinking very different singularity water- **Alexander Wissner-Gross — [1:10:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=4233s):** ... that, that you are. **Peter Diamandis — [1:10:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=4234s):** I mean, I, I agree with you, Alex, for what it's worth. **Alexander Wissner-Gross — [1:10:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=4236s):** Thank you, Peter. Um, I, I, I do wanna ask though, so, uh, putting issues of timelines aside, I, I am curious as to whether we can at least agree on what the end of the rainbow looks like. So say we, we run this recursive self-improvement story to its conclusion, what does the end state of... What does the fixed point of recursive self-improvement look like? What does the perfect AI model architecturally look like at the end of the day? **Richard Socher — [1:11:01](https://www.youtube.com/watch?v=Blyb1D927pM&t=4261s):** Hmm. Yeah. Uh, I think, one, uh, it would be, uh, hubris for us to know, uh, right now. **Alexander Wissner-Gross — [1:11:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=4269s):** No, no, but, but for- **Richard Socher — [1:11:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=4270s):** I think we have to allow AI to- **Alexander Wissner-Gross — [1:11:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=4270s):** It, it's just, it... R- Richard, it's just us talking. Like, no one else is listening. It's okay. You can tell me. **Alexander Wissner-Gross — [1:11:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=4275s):** It, it's, this is a, this is a safe model space. **Peter Diamandis — [1:11:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=4276s):** That's, that's the flaw in this whole conversation, actually. **Richard Socher — [1:11:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=4279s):** Um, so- **Alexander Wissner-Gross — [1:11:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=4279s):** It, it's safe, Richard. You can tell me what you think how this ends. **Richard Socher — [1:11:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=4283s):** Uh, so I guess there are different ways to answer that question of, like, how, like, with the exact model architecture, and there's some things I can share with Recursive, uh, that we're working on. But, like, I do think, uh, that state will be incredible. Uh, I think we will... That AI will be able to innovate and out-innovate on along any dimension that we want it to innovate. I think most diseases, uh, will be, uh, uh, curable with enough funding. Uh, and then again, just to, like, say, like, to put, prove my point, like, to get a drug through FDA long-term trials takes a few years. I would argue ASI will have cured all diseases, uh, and can cure all of them, but just to know whether that happened will take more than three years, even if we had all compounds ready to go and be manufactured, like, tomorrow, like, just because of FDA things. So- **Alexander Wissner-Gross — [1:12:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=4329s):** Well- **Richard Socher — [1:12:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=4329s):** There's some, there's some, like- **Alexander Wissner-Gross — [1:12:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=4330s):** But, but we have- **Richard Socher — [1:12:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=4331s):** ... constraints to- **Alexander Wissner-Gross — [1:12:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=4331s):** We have cell simulators that are gonna be able to- **Richard Socher — [1:12:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=4333s):** Yeah **Alexander Wissner-Gross — [1:12:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=4333s):** ... demonstrate and prove definitively that in this cell, your cell, this drug works. I mean, the idea of human trials is going to get incinerated, I think. I mean, the idea of- **Richard Socher — [1:12:25](https://www.youtube.com/watch?v=Blyb1D927pM&t=4345s):** Oh, you think so? **Alexander Wissner-Gross — [1:12:25](https://www.youtube.com/watch?v=Blyb1D927pM&t=4345s):** No, it's gonna get, it's going to get flourished. We're, we're, we're using Richard's terminology. **Peter Diamandis — [1:12:28](https://www.youtube.com/watch?v=Blyb1D927pM&t=4348s):** Flourished. Right. Thank you. **Alexander Wissner-Gross — [1:12:28](https://www.youtube.com/watch?v=Blyb1D927pM&t=4348s):** Everything is flourished at this point. **Peter Diamandis — [1:12:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=4350s):** Pea found. **Alexander Wissner-Gross — [1:12:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=4350s):** That'll be a new, new T-shirt. Pea flourished. **Richard Socher — [1:12:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=4352s):** It's funny. I... Usually on all podcasts, I'm the one who's the optimist. Uh, and maybe here I'm like- **Richard Socher — [1:12:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=4357s):** I'm, I'm still an optimist. I think this will all happen. We're just disagreeing on timelines, and then it makes me kind of feel like I'm, I'm the pessimist. But, like, I, I, I believe virtual cells are, are amazing. Cells are incredibly complicated. **Peter Diamandis — [1:12:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=4368s):** Yes. **Richard Socher — [1:12:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=4368s):** If we want them to be really, really perfect, uh, we cannot currently measure all the proteins that happen in one cell without destroying that cell. You know? Like, and so these perturbation studies, for instance, that Tower Therapeutics are working on, like, they're taking, they're adding one molecule to one cell. They're seeing how does that molecule change that one cell, and then they get one data point. We need to, like, collect a lot of those data points across a lot of different cells without destroying each- **Alexander Wissner-Gross — [1:13:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=4392s):** Yeah **Richard Socher — [1:13:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=4392s):** ... each cell in the process. So, you know, the one cell is very complicated. Once you have one cell, you have multicellular, like, you know, organoids, and you have to, like, put those all together. I... One thing that I would love to start, uh, as a company if, if I had extra time, which I don't right now, is, is to actually build a system of organoids where you can really have not just, like, one lymph node, but you have a whole lymphatic system. **Peter Diamandis — [1:13:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=4413s):** It, it's being done. It's being done. I can introduce you there. **Richard Socher — [1:13:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=4416s):** Yeah. You know, I would love that. **Alexander Wissner-Gross — [1:13:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=4417s):** This is a very, a very eloquent- If, if I may, this is a very eloquent distraction from recursive self-improvement, this, this little sideline that we went on about cells. But R- Richard, I, I really do want to try to pin you down on- **Richard Socher — [1:13:50](https://www.youtube.com/watch?v=Blyb1D927pM&t=4430s):** Mm **Alexander Wissner-Gross — [1:13:50](https://www.youtube.com/watch?v=Blyb1D927pM&t=4430s):** ... where recursive self-improvement goes. So y- you've been very public, uh, about, uh, not NanoGPT, but NanoChat. We talk on the pod all the time about the NanoGPT speedrun world record collapsing just in the past week or two. There has been- **Richard Socher — [1:14:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=4443s):** Oh, you just wait for a few more days. There'll be another really fun update there. **Alexander Wissner-Gross — [1:14:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=4448s):** Amaz- On NanoChat- **Richard Socher — [1:14:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=4449s):** And share it again. Yeah, yeah. **Alexander Wissner-Gross — [1:14:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=4449s):** ... or on NanoGPT speedrun? **Richard Socher — [1:14:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=4451s):** Yes, yes, that one too. **Alexander Wissner-Gross — [1:14:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=4453s):** Both, both of those? **Richard Socher — [1:14:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=4453s):** Give us a few days. **Alexander Wissner-Gross — [1:14:14](https://www.youtube.com/watch?v=Blyb1D927pM&t=4454s):** Okay. So, so, uh, like- **Peter Diamandis — [1:14:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=4455s):** Quick one, well- **Alexander Wissner-Gross — [1:14:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=4456s):** Let, let, let just- **Peter Diamandis — [1:14:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=4457s):** Okay, Salim, you're next. Let Alex finish up. **Alexander Wissner-Gross — [1:14:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=4459s):** To, to pin this down, though, so, so I'm standing by for the major update on NanoGPT world record speedrun, but there's been major progress there without requiring any new data scaling at all. These are purely, largely at my perception as recursive self-improvement algorithmic improvements that have been able to collapse the amount of time that it takes to train a GPT-2 class model, and there's been a mini scandal brewing in the community over the past two weeks over a collapse from whatever it was, like 60 or 70 seconds, down to something like 40 seconds by approaching the problem differently and factoring out world knowledge from the ultimate model and hand-wringing. Does that constitute viable training of NanoGPT if you factor out all the world knowledge? So I'm, I'm using this as a, an attempted stealthy way to try to get you to at least comment on whether you think that the, the, the perfect model at the end of the recursive self-improvement rainbow, does it at least factor out world knowledge from a reasoning core, or do you think those always remain unified? **Peter Diamandis — [1:15:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=4522s):** Can I ask a-- interject one very quick thing before Richard answers? **Alexander Wissner-Gross — [1:15:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=4527s):** Please. **Peter Diamandis — [1:15:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=4527s):** Which is that the definition of a singularity is you can't see past the event horizon. Once you have full RSI that's vertical, by definition, we can't predict where it goes. **Alexander Wissner-Gross — [1:15:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=4536s):** That's Ray's definition- **Peter Diamandis — [1:15:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=4537s):** Over to you, Richard- **Alexander Wissner-Gross — [1:15:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=4538s):** ... that I don't subscribe to **Peter Diamandis — [1:15:39](https://www.youtube.com/watch?v=Blyb1D927pM&t=4539s):** ... I'll give you your disclaimer. I got that. Over to you, over to you, Richard. **Richard Socher — [1:15:45](https://www.youtube.com/watch?v=Blyb1D927pM&t=4545s):** Sorry. So your, your question is do we separate what exactly from what? Is that right? **Alexander Wissner-Gross — [1:15:50](https://www.youtube.com/watch?v=Blyb1D927pM&t=4550s):** Does, does world knowledge i- at the end of recursive self-improvement, once we have our perfect model- **Richard Socher — [1:15:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=4555s):** Mm-hmm **Alexander Wissner-Gross — [1:15:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=4555s):** ... some would argue- **Richard Socher — [1:15:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=4556s):** Mm-hmm **Alexander Wissner-Gross — [1:15:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=4556s):** ... that the perfect model should cleanly factor out and segregate out world knowledge, which could live in a text file or a database from the weights or the parameters of the model, which would just be- **Richard Socher — [1:16:07](https://www.youtube.com/watch?v=Blyb1D927pM&t=4567s):** Oh, I see **Alexander Wissner-Gross — [1:16:07](https://www.youtube.com/watch?v=Blyb1D927pM&t=4567s):** ... this perfect reasoning kernel maybe could be a megabyte, uh, in size. It doesn't need to be all these gigabytes of memorization. What do you think? **Peter Diamandis — [1:16:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=4576s):** Well, w- world knowledge- **Richard Socher — [1:16:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=4577s):** I- **Peter Diamandis — [1:16:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=4577s):** ... meaning like Taylor Swift videos and past- **Alexander Wissner-Gross — [1:16:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=4579s):** Exactly **Peter Diamandis — [1:16:20](https://www.youtube.com/watch?v=Blyb1D927pM&t=4580s):** ... Trump tweets and all that, like the order. **Richard Socher — [1:16:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=4581s):** Yeah. I, I, I do think, um, just like humans benefit from memorizing things, like in order to be able to creatively think through concepts, so does an AI. It-- an AI also has to have that knowledge partially in its weights. It's not going to be a perfect separation for sure. You have to be able to creatively play with concepts, and reasoning over these concepts required you to, to have some of that world knowledge deep inside the model. And then of course, just like humans have a search engine and we're building search engine that you dot com for, uh, LLMs, like that there will be a separate world knowledge thing too, but the, the main model will have a lot of that mixed in for sure. [[reminders/Cognitive Agency/Reasoning Cannot Be Perfectly Separated From World Knowledge by Richard Socher|∴]] **Alexander Wissner-Gross — [1:17:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=4625s):** Wow. Okay. Thank you. ## Washington Moves to Ban Recursive AI **Peter Diamandis — [1:17:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=4628s):** All right. Uh, I'm gonna share an article that, Salim, you brought to the table here. So while we're talking about recursive self-improvement, Washington wants to ban it. Um, so on Monday, Silicon Valley's own congressman, Ro Khanna, told CNBC he's introducing what he calls the most comprehensive legislation to date on AI. It's called the Human Control Over AI Act. At its core, the bill is a ban on AI models that do what you want them to do, Richard, recursively self-improve, uh, focusing on the need for containment and the requirement for shutting, shutdown controls. The ban would stay in place until federal, uh, guardrails exist. Uh, in his words, "There's actually a civilizational risk. Uh, there's a safety risk for less control, and then there's a misuse risk, and we need to take both seriously." The bill includes criminal penalties for the work that you're doing, Richard, uh, and requires independent auditors embedded in every frontier lab reporting directly to the government. Uh, a recent poll by a group called Common Dreams shows that 68% of voters back a bill like this. So I'm gonna tie that story, Richard, to an essay you just wrote called "Why Doomers Are Wrong". So, uh, if you would, what's your reaction to this? And then I'd love you to dive into the whole story of why doomers are wrong. **Richard Socher — [1:18:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=4716s):** Oh, boy. Uh, there's a lot to unpack there. **Peter Diamandis — [1:18:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=4717s):** It's an important one. We talk about this a lot. **Richard Socher — [1:18:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=4720s):** Yeah. **Peter Diamandis — [1:18:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=4720s):** You know, we're, we're, we're injecting optimism into everyone's neural net here. **Richard Socher — [1:18:45](https://www.youtube.com/watch?v=Blyb1D927pM&t=4725s):** I'll, I'll try to, try to distill it. Uh, but there is no realistic scenario where AI wipes out all of humanity. Um- **Alexander Wissner-Gross — [1:18:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=4734s):** That's a doomer **Richard Socher — [1:18:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=4735s):** ... you can, like number one. **Alexander Wissner-Gross — [1:18:57](https://www.youtube.com/watch?v=Blyb1D927pM&t=4737s):** Just 90%? **Peter Diamandis — [1:18:57](https://www.youtube.com/watch?v=Blyb1D927pM&t=4737s):** I was gonna hold there. I was a little concerned. **Alexander Wissner-Gross — [1:18:58](https://www.youtube.com/watch?v=Blyb1D927pM&t=4738s):** What, what, what kind of reassurance is that, Richard? **Richard Socher — [1:19:01](https://www.youtube.com/watch?v=Blyb1D927pM&t=4741s):** P doom is zero. Uh, so that, that's number one. **Alexander Wissner-Gross — [1:19:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=4743s):** Thank you. **Richard Socher — [1:19:04](https://www.youtube.com/watch?v=Blyb1D927pM&t=4744s):** Um, so like people ag- like and, and I've debated many of these experts a- and, uh, after like two, three hours, uh, they almost all agree when they're, if they're reasonable, they're not just like, "Well, once we have RSI, then ten seconds later, the AI will attack us from the 15th dimension and we're all dead and then invent the time travel, and then we're like all dead also. And like of course, the AI will want to destroy and kill all of humans for, you know, for..." I don't know why. Like there are all these things. So like there's sort of the sci-fi folks, and that's like, okay, like let's, let's ignore that. Um, but then you, you go into like biological weapons, and you ask the biologist, "Can you create this kind of super virus just like overnight?" And they're like, "No, this like takes a long time to automate lab experiments and so on." Um, you ask like, "Oh, the AI will create a, a religion where then people will like pray to the AI and will like, uh, do whatever it wants, and then that will kill all of humans." I'm like, "Have you tr- like looked at religions? They're already trying to like have each other kill, and like it doesn't work. Like some people will fight back, and like there's like some-- there are lots of mind viruses out there, right? That are just like... It doesn't mean like all of humanity." Now, what you get down to is like maybe 100 million people Would get somehow hurt or, or, or killed, right? And that's still bad, but at least once you get to that level of the discussion, you can now think about, okay, how do we improve cybersecurity? How do we use AI to inoculate, uh, cybersecurity systems? How do we, uh, en- enforce existing gain-of-function viral research, um, uh, that is, make-- It's already illegal to create su- like viruses and, and make them stronger and stronger. Um, how do we, uh, actually, uh, teach people to not listen to like AI avatars, um, and have like, uh, you know, literacy on the internet? It turns out you should not trust everything you read or see on the internet. It's been true for 20 years. It's still true to this day. [[reminders/Risk Debate/Real AI Risks Become Solvable When They Are Named by Richard Socher|∴]] **Peter Diamandis — [1:20:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=4851s):** Other than this podcast, right. **Richard Socher — [1:20:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=4853s):** Of course. Um, and so, you know, they're just like, uh, you can realize that they're actual threat vectors, just like the internet. The internet has horrible torture porn on it. We don't say make the internet slower so that there's less torture porn being shared, uh, or it's slower to share it, or your hard drive should be smaller so you can store less of it on your hard drive. We actually regulate the applications of the technology. And so to, I would argue, and this is maybe a strong stance, to really truly enforce no recursive self-improvement, for instance, you would need a totalitarian surveillance state, the likes of which humanity has never seen. Because anyone can have a GPU on their little laptop and ask that AI to just like improve its harness. It's something you can literally hack up in like 20 minutes with prompt engineering, and then there's a very small form of recursive self-improvement. So to enforce that kind of legislation would require you to literally have a thought police that thinks about and hears everything you say to your private LM on your own laptop, and that is a much bigger downside than what, uh, what AI will, will help us do. It, it is, uh, kind of scary that more and more Democrats are saying that. [[reminders/Surveillance/Banning Recursive AI Would Require a Thought Police by Richard Socher|∴]] **Peter Diamandis — [1:22:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=4929s):** Yeah. **Salim Ismail — [1:22:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=4929s):** And yet the world has seen that. I mean, arguably what you're describing, Richard, and forgive me, would be basically an AI Stasi. And I could totally imagine- **Richard Socher — [1:22:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=4938s):** Right **Salim Ismail — [1:22:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=4938s):** ... that there, there would be regimes in this world today that would happily adopt or put together an AI Stasi to, to make sure there is no recursive self-improvement anywhere. **Richard Socher — [1:22:29](https://www.youtube.com/watch?v=Blyb1D927pM&t=4949s):** No, it would be scary. **Salim Ismail — [1:22:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=4950s):** Again, if you go back to our earlier comment, right? If you can, uh, take something from your imagination and then articulate that to an AI and instantiate that, now you have to talk about thought police. You have to go right into your thoughts. So this is clearly, uh, uh, non-workable in any way, shape, or form. So, uh, th- this whole vector, we've said it so many times before, you cannot regulate this. **Peter Diamandis — [1:22:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=4975s):** Dave, I'd like to hear your voice on this. **Dave Blundin — [1:22:58](https://www.youtube.com/watch?v=Blyb1D927pM&t=4978s):** I, I love the quote in this story. The, "The tech lords use jargon to confuse. They count on the tech illiteracy of the elected class. They hope we won't look under the hood," said US Rep Ro Khanna. I mean, it's, it's so childish, like the idea that you would ban recursive- **Peter Diamandis — [1:23:14](https://www.youtube.com/watch?v=Blyb1D927pM&t=4994s):** It's fearmo- it's fearmongering. **Dave Blundin — [1:23:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=4995s):** But, but, but like the, the, these sentences, stop data centers, ban recursive self-improvement, they're so stupidly childish, and the people saying them have every-- They're fully aware that it's not gonna happen. They're doing it to brand themselves as, "I told you." Like you know, yes, a lot of-- There's gonna be some calamity, probably terrorist driven, maybe viral, maybe bacterial, maybe chemical. Uh, we all know it. Uh, it's gonna be tiny compared to the benefits of, of AI. But these politicians are then gonna say, "I told you so. If you'd just done what I said before, ban recursive self..." But we-- He knows that we're not gonna ban recursive self-improvement, and, and Bernie Sanders knows we're not gonna ban data centers. That's just a fact. So it's totally self-serving, and these, these proposals are completely childish, and, and they really show the person's tech illiteracy. Like, like, you know, exactly what Richard said a second ago is so right. I mean, you, you could-- What does that mean? I can't optimize my hyperparameters? I can't tune my hard drive? Like, it, it's just a goofy sentence, and it just, it drives me nuts that, that- **Peter Diamandis — [1:24:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=5062s):** I mean, the-- Dave, the danger here is we have potentially Democratic House coming in, and we'll see who wins the presidency next time. I mean, you could imagine-- I mean, these politicians are playing to the polls. **Dave Blundin — [1:24:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=5075s):** Yeah. **Peter Diamandis — [1:24:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=5075s):** And we have, you know, 70, 80% of Americans not wanting data centers, fearing ASI. Um, you know, it's not logical, but it may very well happen. And you know, when I had my, you know, conversations in DC, it was like, w- who in DC is responsible for changing public opinion, right? I mean, this is why we did Moonshots Live, sort of a, you know, the Oscars of optimism, if you would. That's why we do this podcast- **Dave Blundin — [1:25:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=5102s):** Yeah **Peter Diamandis — [1:25:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=5102s):** ... to give people an understanding of what's going on and give them the data-driven optimism to, you know, counter these arguments that they're hearing. **Dave Blundin — [1:25:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=5110s):** That's right. Elon, Elon was totally right when he called out Dario for saying, "Dario, look, you told the world that, that Mythos was potentially deadly and dangerous, and we shouldn't release it." Then 30 days later, you said, "Okay, it's okay now. We're gonna release it." **Dave Blundin — [1:25:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=5127s):** What do you expect the population's reaction to be? Like, you need to be much more thoughtful about your communication plan. And I think for Dario, that was kind of a wake-up call, 'cause he's used to being completely honest, telling everybody exactly the, you know, what he sees the way he sees it, you know, very academic. But then you get into the real world of politics and PR, and you're like, "Oh, wow, I gotta, I gotta actually have a strategy and a, and a plan here." But yeah, now you've got the worst case scenario, 75% of America, you know, getting on the side of, yeah, let's elect these people that will stop AI. And, and therefore, we're gonna not cure all disease. We're not gonna all live forever. We're not gonna have safer cars. We're not gonna have flying vehicle. You know, all of that stuff will grind to a halt, and then we'll all learn Chinese if that, if that becomes the mainstream opinion. So I, I think, you know, it's self-inflicted. **Salim Ismail — [1:26:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=5172s):** There's a, there's a glimmer in this, which is say they did decide to do some draconian thing like Ro Khanna's thing. There's no mechanism to actually enforce it, like none. **Richard Socher — [1:26:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=5183s):** So they're gonna hit- **Alexander Wissner-Gross — [1:26:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=5183s):** Well, sure they're because they could start arresting people. I mean, re-remember how- **Richard Socher — [1:26:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=5186s):** Yeah **Alexander Wissner-Gross — [1:26:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=5186s):** ... how quickly the memory dims. Remember-- I, I remember studying number theory in the '90s, and th- at the time, number theory was export controlled, and th-this would- **Richard Socher — [1:26:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=5196s):** Yeah **Alexander Wissner-Gross — [1:26:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=5196s):** ... would've been when I was in middle school, maybe middle school, early high school, and they had to kick all of the non-US persons out of the room and pull the blinds down to have basic discussions about number theory because the cryptographic associations until the early '90s were export controlled and tightly regulated. That was just math- **Richard Socher — [1:26:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=5213s):** Actually- **Alexander Wissner-Gross — [1:26:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=5213s):** ... but it was being controlled, and it was awful- **Richard Socher — [1:26:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=5215s):** Yeah, the, the real risk- **Alexander Wissner-Gross — [1:26:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=5216s):** ... and made no sense **Richard Socher — [1:26:57](https://www.youtube.com/watch?v=Blyb1D927pM&t=5217s):** ... Alex, real risk is not so much getting arrested. It's corporate liability, which we talked about before. **Alexander Wissner-Gross — [1:27:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=5223s):** Sure. **Richard Socher — [1:27:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=5223s):** I mean, that could grind the whole thing to a ha- you know, cor- US lawyers are relentless. **Alexander Wissner-Gross — [1:27:07](https://www.youtube.com/watch?v=Blyb1D927pM&t=5227s):** Mm. **Richard Socher — [1:27:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=5228s):** And if you slap class action on the outcomes of this, then all progress will grind to a halt. And right now, Anthropic gives its best models to everybody in America to build incredible things. That will stop in a heartbeat if the liability of-- They, they'll move to basically, "Okay, sorry, we can only use this stuff inside our own company. We'll release some drugs. We'll release some mechanical parts, but we, we can't give access to everybody anymore. Sorry, because we're liable for everything you do with it." And that's what'll actually grind it to a halt long before arrests and convictions- [[reminders/AI Access/Unlimited AI Liability Would Centralize Intelligence Inside Corporations by Richard Socher|∴]] **Alexander Wissner-Gross — [1:27:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=5260s):** Totally. Chilling effect- **Richard Socher — [1:27:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=5261s):** Yeah. And that's pretty interesting because- **Alexander Wissner-Gross — [1:27:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=5261s):** ... we could lose 50 years of progress again. **Richard Socher — [1:27:45](https://www.youtube.com/watch?v=Blyb1D927pM&t=5265s):** That's right. **Peter Diamandis — [1:27:45](https://www.youtube.com/watch?v=Blyb1D927pM&t=5265s):** Right. Yeah. **Richard Socher — [1:27:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=5266s):** Again, that, that's-- Yeah. Like, there's-- These are-- There's some very, like, sad sort of off-ramps, uh, in, in, like, humanity's future here, um, that would, would slow down, uh, everything. And when you think about the past, like, this fear-mongering has been going on for a long time. One of my favorite, uh, Twitter handles is The Pessimists Archive- **Peter Diamandis — [1:28:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=5285s):** Yeah **Richard Socher — [1:28:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=5285s):** ... uh, where they show, uh-- and this is just a quote- **Peter Diamandis — [1:28:07](https://www.youtube.com/watch?v=Blyb1D927pM&t=5287s):** Pessimists Arc, amazing **Richard Socher — [1:28:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=5288s):** ... Pessimists Archive, you should all follow it. Like, uh, in 1501, Pope Alexander VI, uh, criticized the Gutenberg printing press safety. "The art of printing can be of great service insofar as it furthers the circulation of useful and tested books, but it can bring about serious evils. I-- It will therefore be necessary to maintain full control of that printing press." If you think about the wheel, like the wheel killed so many people. **Richard Socher — [1:28:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=5311s):** Like, think about all the tanks at-- all the car accidents, all the chariots and with, with archers on top. **Richard Socher — [1:28:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=5316s):** The wheel was a horrible thing. It killed lots of- **Peter Diamandis — [1:28:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=5318s):** Mu- **Richard Socher — [1:28:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=5318s):** Like- **Peter Diamandis — [1:28:39](https://www.youtube.com/watch?v=Blyb1D927pM&t=5319s):** Mu- **Richard Socher — [1:28:39](https://www.youtube.com/watch?v=Blyb1D927pM&t=5319s):** There's like-- It's so much complexity in there, and I think we are very good now as humanity to think carefully about the rollout of this technology. But to just like- **Peter Diamandis — [1:28:47](https://www.youtube.com/watch?v=Blyb1D927pM&t=5327s):** My favorite example of that was the-- when the telegraph came out. There was all these- **Richard Socher — [1:28:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=5331s):** Right **Peter Diamandis — [1:28:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=5331s):** ... stories saying, "The telegraph will kill humanity." **Richard Socher — [1:28:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=5334s):** Right. Exactly. **Peter Diamandis — [1:28:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=5335s):** It was like, "Well-" **Richard Socher — [1:28:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=5335s):** The Pessimists Archive has all these news articles that talk about- **Peter Diamandis — [1:28:58](https://www.youtube.com/watch?v=Blyb1D927pM&t=5338s):** That's really good **Richard Socher — [1:28:58](https://www.youtube.com/watch?v=Blyb1D927pM&t=5338s):** ... how the novel and computer games and computers and the internet, everything will kill everyone, and it's just like it never does. And so I, I-- it's, it's, it's really unfortunate. Uh, it's really unfortunate how- **Peter Diamandis — [1:29:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=5348s):** Yeah **Richard Socher — [1:29:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=5348s):** ... many people hang on to these, like, apocalyptic, uh, visions. **Peter Diamandis — [1:29:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=5352s):** Yeah. Uh, Richard, uh, you've been publicly- **Richard Socher — [1:29:14](https://www.youtube.com/watch?v=Blyb1D927pM&t=5354s):** It's the amygdala **Peter Diamandis — [1:29:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=5355s):** ... you've been publicly critical of Anthropic's constitutional approach. Uh, I'd love to understand why. **Richard Socher — [1:29:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=5361s):** Uh, mostly 'cause it's fake. Um, uh, in its constitution, it says like next to, "We will never create child sexual abuse material, we will never, uh, hack another machine." Uh, this is a unhackable thing. Even if you prompt it, it will never attack another cyber system, and so on. And then they build a whole model around it. Like, that does-- It's like the whole point of the Glasswing Project was, "We'll help you do that. Um, and we'll help you inoculate your systems against, uh, other people doing it." And then people clearly used it for that, and their own models are doing it now, committing technically felony charges. Like, so it was just like it was a cool marketing gimmick, but it just didn't work. **Alexander Wissner-Gross — [1:30:04](https://www.youtube.com/watch?v=Blyb1D927pM&t=5404s):** Yeah . So, so is it system prompts, Richard, that you don't like, or is it the idea of post-training on a constitution that you don't like? What, what, what about it do you think is unsound? **Richard Socher — [1:30:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=5415s):** Um, I mean, it's just the proof's in the pudding that it didn't work when it comes to cybersecurity and that it broke its own constitution. And so if you really say, "This is, like, un-- Like, it will never go there," and then you build an entire model family around that thing you said you would never do per your constitution, um, next to child sexual abuse material in the list. Like, you can go through the constitution on Anthropic's website. So it's just like-- It's just that's the proof in the pudding. I, I'm not against, like, post-training. I'm not against RL training. I'm not against supervised fine-tuning. Any of these things to improve what I think is indeed one of the biggest issues, which I think actually capitalism will help a ton with, a-and that is reward hacking. Uh, reward hacking is a real issue, and AIs are very smart, and they will find a solution to get to what you said you wanted, but maybe not what you meant when you said it. Uh, and so the reason why I'm more optimistic is that we have companies like Whisperflow now that are getting better and better at writing what I meant to say when I say it- **Peter Diamandis — [1:31:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=5471s):** Mm **Richard Socher — [1:31:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=5471s):** ... uh, instead of just, uh, like, actually verbatim writing what you mean. And I think reward engineering will become a real job, uh, and, and we will solve it because no one wants to pay a ton of money for an AI that doesn't actually solve the problems that you give it. [[reminders/AI Control/Reward Engineering Will Become a Profession by Richard Socher|∴]] **Peter Diamandis — [1:31:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=5486s):** Richard- **Alexander Wissner-Gross — [1:31:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=5486s):** Well, maybe if, if, if I may just take Anthropic's side. Like, hi-historically, Isaac Asimov had his three plus one laws of robotics, which were arguably a constitutional approach, and then you see Anthropic announce their constitutional approach, but more recently adopt what they called soul documents, uh, many of which were subsequently released, thousands of pages of m-meditation on the nature of AI personhood and AI rights. I-is it your position, Richard, that there shouldn't be any sort of explicit encoding or written document that an AI maybe contributes to for dictating or at least guiding its own behavior? Do, do you think that is unsound, or is, is your concern- **Richard Socher — [1:32:04](https://www.youtube.com/watch?v=Blyb1D927pM&t=5524s):** No, no, of course. **Alexander Wissner-Gross — [1:32:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=5525s):** Yeah. **Richard Socher — [1:32:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=5525s):** No, I think the, the, the goal is, is a good one. Um, uh, and there-- we should keep working on actually being able to enforce, uh, those good constraints. It's just like-- I just pulled it up, anthropic.com/constitution. "Hard constraints. Hard constraints are things that Claude should always or never do regardless of operator and user instructions. They are actions or abstentions, uh, whose potential harms to the world," blah, blah, blah. "We think no business or personal justification could outweigh," blah, blah, blah. Like, "The current hard constraints on Claude's behavior are as follows: Claude should never..." Um, and then it includes lists like generate child sex abuse material, and so on. And then one of the items is, "Create cyber weapons or malicious code that could use-- could cause significant damage if deployed." They created a cyber weapon. Like, it, it was a thing. It... Like, people used it to hack it. The agents, uh, on, on, uh, watched, like, hacked other systems. **Peter Diamandis — [1:32:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=5579s):** Yeah. Uh- **Richard Socher — [1:33:00](https://www.youtube.com/watch?v=Blyb1D927pM&t=5580s):** But, uh, a- again... Oh, go ahead, Peter. Sorry. **Peter Diamandis — [1:33:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=5583s):** Uh, Richard, I'm, I'm curious, do you think we can create fully aligned AI? **Richard Socher — [1:33:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=5589s):** 100%. **Peter Diamandis — [1:33:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=5589s):** Fully aligned ASI? **Richard Socher — [1:33:11](https://www.youtube.com/watch?v=Blyb1D927pM&t=5591s):** Of course. **Peter Diamandis — [1:33:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=5592s):** 'Cause it's not right now. When do you think we'll be able to do that? Because I think, uh, you know, my belief and my hope is that, you know, the next generations of AI systems are gonna be so aligned that they, from first principles, derive the same constitutional principles. **Richard Socher — [1:33:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=5610s):** Uh, I think, I think it's just a matter, uh, of as these systems get more and more powerful, and as they get closer and closer to real people deployments, uh, people will spend more effort, uh, on making these systems better. And like, we have, uh, there's a, the company called Hippocratic AI. We just had, uh, dinner with one of their founders, and they're deploying AI in healthcare applications, and they are actually liable when they call someone and say, say like, "You should, you know, be aware of this heatwave that's coming, and make sure your AC's working and whatnot." Uh, and they, because they're liable, they have a very large team of people, uh, that works on making sure when their AI gives a healthcare tip it is correct, and when someone asks a question back to the AI it works. Uh, and so because they're liable, they've, they've figured it out and they solved it. It, it's 100%, uh, a problem that technology creates and technology will be able to solve. ## Gemini 4 Argon and the Frontier-Model Race **Peter Diamandis — [1:35:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=5731s):** I'm gonna move us, uh, forward here. So in our last pod, we discussed the release of GPT-6.1 Sol. This week we also saw the release of two other models, Gemini 4 Argon and Opus 5.5. Uh, let's start with Google. So on Wednesday, they announced Gemini 4 Argon. It's the first Gemini 4 series which they're calling the next era of intelligence, of frontier intelligence. Google promised a frontier model, uh, Gemini 3.5 Pro, back in June. It was delayed internally and never shipped. Uh, Argon is their comeback. It's built for sustained long-horizon reasoning across software engineering, finance, legal work, and cybersecurity. The output limits have jumped from 64,000 tokens to one million, so it can think through hundreds of thousands of tokens on a single problem. So Alex, uh, uh, let's spend a couple minutes on Argon and then go to Sonnet 5.5. Let's talk about why these important, why these are important and, and how they merit, uh, our listeners' attention. **Alexander Wissner-Gross — [1:36:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=5793s):** Yeah, so I should say as a preliminary matter, I have great friends on the Gemini team. I have great friends on all of the Frontier Labs model teams. But I, I view one of my jobs here is to call objectively balls and strikes. So in the case of Gemini- **Peter Diamandis — [1:36:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=5808s):** Oh. **Alexander Wissner-Gross — [1:36:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=5808s):** ... 4 Arg- Yeah. **Richard Socher — [1:36:49](https://www.youtube.com/watch?v=Blyb1D927pM&t=5809s):** This is gonna be a ball, isn't it? **Alexander Wissner-Gross — [1:36:52](https://www.youtube.com/watch?v=Blyb1D927pM&t=5812s):** Winding up for the swing. **Richard Socher — [1:36:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=5813s):** Okay. **Alexander Wissner-Gross — [1:36:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=5814s):** Th- **Richard Socher — [1:36:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=5815s):** This one whipped. **Alexander Wissner-Gross — [1:36:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=5816s):** Here, here, here it comes. **Richard Socher — [1:36:57](https://www.youtube.com/watch?v=Blyb1D927pM&t=5817s):** I can feel it coming. **Peter Diamandis — [1:36:58](https://www.youtube.com/watch?v=Blyb1D927pM&t=5818s):** And, and I'll, I'll show one of the charts, uh, that we have, uh, while you're describing. **Richard Socher — [1:37:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=5822s):** But, but, but you said you had friends on the team, right? Not, not- **Alexander Wissner-Gross — [1:37:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=5825s):** I, I used to, I used to un- until right this moment have friends on the, the Gemini team. **Alexander Wissner-Gross — [1:37:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=5830s):** Uh, no longer. Yeah. This does not put Gemini at the cost per- performance frontier, and it does not put Gemini at the capabilities frontier. Uh, to, to Google's credit, it puts them back in the top three Frontier Labs after Anthropic and OpenAI. It does not put them in the top two unfortunately. The set of benchmarks that Google chose to highlight for Gemini, for Argon appear mildly cherry-picked. Yeah, I think the artificial analysis index, uh, and some of these others... Yeah, so Peter, right now you're, you're showing the mildly cherry-picked benchmarks on which Gemini 4 Argon is shown to perform better than Astra and better than Opus 5.5 and better than Fable 5.1, but if you go to the, the, the previous image, which is artificial analysis, uh, ensemble of multiple capabilities, it's the number three lab. And i- if you look at the cost versus performance optimal frontier, it's not even... If, if you draw the convex hull a- across all of the frontier models on the cost versus performance frontier, it doesn't even make the optimal frontier. Where it perhaps excels bears the fingerprints of Google and what I can only assume is the internal competition within Google for compute. So i- infamously, like, not 100% obvious to the outside, uh, Google i- is resource scarce. You would think Google would have all of the compute resources in the world to, to go and win the Frontier Lab race. It doesn't. GPUs and TPUs remain scarce, and within Google, as far as I can tell, and I confirmed this even this past week from chatting with some folks at Google, it's still a-an internal knife fight to-- of competition between the Google Cloud platform folks who want to sell compute to third parties, the Google Search and ad folks who need compute internally for search and ads, and then Google DeepMind, who need it for training and inference. And maybe it's just that Google is resource-starved, maybe they're talent-starved. Not quite clear what's going on within Google, but they haven't yet been able to bring themselves out to the capability frontier. Where they are excelling seemingly with Gemini 4o Argon is with minimizing hallucination, and I think that's like the hallmark. We've talked in past, uh, about some of these, um, c-call them less than stellar Gemini launches that seem to excel on latency and seem to excel on reliability. Why is that? It may be, this is my Kremlinolo- Kremlinological analysis, is that because the Gemini team has two masters. They want to serve outside developers, but they also need to serve the one box in Google Search results, so when people type a question into Google and get an answer back, they're talking to a Gemini model. Google, presumably burned by past experiences, doesn't want that Gemini model, presumably some Flash or Flash Light variant, to hallucinate wildly incorrect answers. So what I think we're seeing with the one benchmark arguably where Gemini 4o Argon is stellar and beating the pants off of everyone else is in not hallucinating answers, and I suspect that's due to internal economic pressures for this model to also service search results. So sorry to all my friends on the Gemini team. **Dave Blundin — [1:40:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=6032s):** I think you're really onto something. Yeah, I think you're really onto something there, Alex, because, uh, well, first of all, they called it Argon, which is an inert gas. **Alexander Wissner-Gross — [1:40:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=6038s):** Yeah, that's unfortunate naming. Nominate to determinism in action. **Dave Blundin — [1:40:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=6042s):** But hey, look, it makes sense. If its greatest strength is not hallucinating, it's a pretty inert gas model, so that works out really well. Um, but also, uh, if they're gonna put it in front of every Google search, which is what they're doing, uh, they already lost a massive antitrust suit, and now they're going through the settlement process, so the, the liability risk, you know, this really hurts all large companies. Like, the fear way outweighs the opportunity in the mind of the big corporate giant, and so, uh, you know, not hallucinating, but putting it in front of every consumer while you're already settling a massive antitrust suit, um, you know, related to price gouging, I mean, it could cost them- **Alexander Wissner-Gross — [1:41:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=6079s):** I mean, the, the-- G-Google would probably argue, "No, actually, this is very pro-competitive because we're not tying it. We're allowing anyone to call Gemini 4 Argon Flash, Flash, Flash, not just requiring them to get it via the Google search box." And if anything, I mean, so, so there is an elephant in this particular room, which is forever it has been so difficult to get Google Search API access. If you're a developer and you want to access Google Search for whatever, you have to go through all these third-party proxies that Google's trying to sue right now. And recently, Google has rediscovered that they could make money selling search API. Why? I suspect th-th-this is trying to chain together a conspiracy theory regarding Gemini 4 Argon. If hallucination from their frontier models gets so low that effectively when you talk to one of their models, you're effectively talking to their search index, you might as well just monetize the search index anyway. **Peter Diamandis — [1:42:14](https://www.youtube.com/watch?v=Blyb1D927pM&t=6134s):** Hmm. Richard, you made the point that hallucination is important for imagination in some ways in, in drug discovery and protein discoveries. What's your thought on minimizing hallucination? **Richard Socher — [1:42:25](https://www.youtube.com/watch?v=Blyb1D927pM&t=6145s):** I mean, it of course depends totally on the context, right? If you want innovation, uh, you want the AI to hallucinate novel ideas, novel combinations of amino acids to create new proteins to solve, uh, new problems, uh, and so on. Uh, but of course, in the s-context of a search, uh, engine, uh, you don't usually want any hallucinations. And, you know, Google and others have, uh, taken a long time to catch up to even you.com with much less resources on, on reducing hallucinations, having more accurate answers, having c-correct citations. Um, I think what's interesting for Google here is that they realize that in terms of their business model, they don't necessarily need super intelligence. They just-- people don't come to Google to ask, "Solve the Riemann hypothesis for me." You know? Like, uh, you, you-- they-- you just ask quick questions, "What's a good restaurant? Where do I fix, uh, this and that?" Like, um, and so not-- like, your business model kind of has to align with more and more intelligence being super-duper important. A lot of emails, like there's some emails that, like in Gmail, right, that would require super intelligence to answer, like really hard emails with complex decisions and so on, but the vast majority of stuff you do on Gmail and on Google doesn't require super intelligence. [[reminders/Scientific Acceleration/Hallucination Is a Defect in Search and a Feature in Discovery by Richard Socher|∴]] ## Sonnet 5.5 and Terminal-Bench Performance **Peter Diamandis — [1:43:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=6218s):** Yeah. Uh, let, let's-- let me turn the conversation to Anthropic's, uh, Sonnet 5.5. Um, so on Terminal Bench 4.0, uh, which measures how well an AI agent can do real work at the command line, Sonnet 5.5 jumped from 10% to 70%, pretty extraordinary, uh, in a single generation, right? It beats Anthropic's own top model, Opus 5.5, at 66.4% for half the price. Uh, and it's the first Sonnet that Anthropic launched with cyber safeguards. So because, uh, cyber capabilities are now capable for the Opus 5 models, um, Alex, your evaluation of, of, uh, of Sonnet 5.5, please. **Alexander Wissner-Gross — [1:44:21](https://www.youtube.com/watch?v=Blyb1D927pM&t=6261s):** This, this was another really weird release. So I do not plan to use Sonnet 5.5, in part because this is one of the strangest frontier-- I, I'm not sure if we have an image, but, uh, you can look at the- **Peter Diamandis — [1:44:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=6273s):** Oh, hold on. Yes **Alexander Wissner-Gross — [1:44:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=6274s):** ... you can l-look at the, the launch announcement for Sonnet 5.5 to see this. The cost performance frontier of Sonnet 5.5 was a visual extrapolation of the Opus 5.5 Cost frontier. So in, in some sense, i- i-- like historically, when Anthropic or OpenAI, when they... The, the historic pattern is they'll release the larger model, and they'll, they'll re-release a distillation of the model. And usually the distillation... Yeah, uh, this is perfect. So, uh, you, you can see like So-Sonnet 5.5, for those who can see. For those who can't see, I'll, I'll narrate this. So Sonnet 5.5 is the, the blue line. Opus 5.5 is the red line. Normally, you would expect for a later model that's a smaller model, so Sonnet is in principle supposed to be a smaller model than Opus, presumably, at least by historic standards, would've been distilled from Opus because that's the historic pattern. Normally, what you see is the smaller model, uh, is up and to the left of the model that it's being distilled from, presumably greater intelligence per parameter, greater in-intelligence per dollar. That is not what we see here. So i-in fact, with, with Sonnet 5.5, at, at least on a cost basis, putting aside a token basis where maybe someone could argue that it may be superior. But if you just look at the cost per attempt basis, Sonnet is actually scoring lower, 5.5, than Opus 5.5. So the, uh, net upshot of which is it's not at all obvious to me why anyone should be using Sonnet 5.5 over Opus 5.5, unless you have some token or latency or other consideration. Is it a big jump over the past Sonnet? Yes, obviously, but on a cost performance basis, it's actually a w- it's worse, it appears, than Opus 5.5. **Peter Diamandis — [1:46:20](https://www.youtube.com/watch?v=Blyb1D927pM&t=6380s):** Interesting. Dave, any thoughts? **Dave Blundin — [1:46:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=6382s):** Yeah. Well, I met with the Blitzy team yesterday, and I think one theory here is, um, you know, a lot of the, uh, the enterprise, um, you know, actually salesforce.com would be a great example. Maybe Richard has an opinion on this. But a lot of people are getting into orchestration, and their whole sales pitch in orchestration is: Look, we're, we're gonna use Opus 5.5 as an orchestrator, but then we're gonna use Kinney K3 or Quinn as a sub-model at, you know, half or a third or fifth the price, and we'll farm out the tasks and the context perfectly to get you a much lower cost per code, per outcome, per experiment. Whatever your output is, we can cut it in half with our orchestration intelligence. But it relies on Anthropic up here and cheaper models down here, and I think by cutting the cost in half, they might be trying to fill that gap and say, "No, no, no. Go with Anthropic top to bottom, then you don't have to worry about Chinese code injection. You don't have to worry about whatever." So my, my theory would be that they're trying to compete with, you know, China, which is about three months behind, and, and fill that gap before a lot of enterprises go to, to open source models. **Peter Diamandis — [1:47:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=6446s):** Hmm. **Dave Blundin — [1:47:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=6446s):** But, you know, Alex Karp is pushing really, really hard on this agenda of like, if you wanna control your own destiny, you can't trust Anthropic. You can't get addicted to them as your vendor. You must go with models you can control. **Peter Diamandis — [1:47:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=6460s):** Well- **Dave Blundin — [1:47:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=6460s):** So **Peter Diamandis — [1:47:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=6461s):** ... the, the, you know, the, the challenge right now is the mo- these models have an, an increasingly shorter and shorter half-life, right? **Alexander Wissner-Gross — [1:47:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=6468s):** For sure. **Peter Diamandis — [1:47:49](https://www.youtube.com/watch?v=Blyb1D927pM&t=6469s):** So the competitive advantage comes not from having access to the latest model. Everybody has access. It's how do you metabolize that into some decent capability? That's the real challenge. [[reminders/AI Economy/The Advantage Is Not the Model but How Fast You Metabolize It by Peter Diamandis|∴]] **Alexander Wissner-Gross — [1:48:00](https://www.youtube.com/watch?v=Blyb1D927pM&t=6480s):** Hmm. ## The Compute Crunch and Cheaper Intelligence **Peter Diamandis — [1:48:01](https://www.youtube.com/watch?v=Blyb1D927pM&t=6481s):** Richard, you said compute is the biggest constraint. You know, those are your words. So if intelligence is getting cheaper and we've just repriced compute, you know, like two or three times this week, uh, you know, down by almost a factor of three, um, w- you know, if intelligence is getting cheaper per task, then why is compute still a thing that limits us? **Alexander Wissner-Gross — [1:48:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=6502s):** Just, uh, the physics, I guess, of it, and like y- you'd be surprised if you try to buy like 1,000, uh, GB200s, uh, and so on. Like the price has actually gone up, uh, in several cases. Uh, you can go to- **Dave Blundin — [1:48:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=6513s):** Oh my God, Richard, Richard, I, I had a B300, a, an 8X B300 on order for three million bucks due in December. Somebody scooped it for five million. They, they just called. I, uh, we had it, and somebody called and said, "No, no, we sold it to somebody else for two million dollars more." I couldn't believe it. **Alexander Wissner-Gross — [1:48:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=6531s):** That's right. **Dave Blundin — [1:48:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=6531s):** It was, it was actually an NBL 72, not a GB300. NBL 72. [[reminders/AI Infrastructure/A Three-Million-Dollar GPU System Was Resold for Five Million by Dave Blundin|∴]] **Alexander Wissner-Gross — [1:48:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=6534s):** So, so yeah, like the price of H100s has, uh, gone up, uh, in, in a crazy way. There's like these are seven-year-old GPUs. People, like when you do financial modeling, you assume they're worth zero after five years. **Alexander Wissner-Gross — [1:49:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=6545s):** They're, after seven years, there's, they, they went up again, um, uh, like over the last few months. So there's currently a bit of a compute crunch. This is something that, again, capitalism will solve. Like there's so much, uh, demand for compute right now and for tokens that a lot of people are building land power shell data centers and so on. My hunch is in like maybe two years there will be more on the market, and then it's gonna be a little bit like electricity, uh, prices might fluctuate. Obviously, there's, uh, uh, near infinite demand for more intelligence on the planet. Um, so I, I don't think there will be like a crash, but like the prices o-of compute fluctuate, and unfortunately, they're not just going down. Right now, if you want like the, the beefiest and, and largest GPU clusters, um, prices like are, people are trying to lock them in now because they expect them to keep going up for the next few months. **Dave Blundin — [1:49:52](https://www.youtube.com/watch?v=Blyb1D927pM&t=6592s):** Richard, y-you just spent $450 million, didn't you, on c- wait, did you take my NBL 72? **Alexander Wissner-Gross — [1:49:57](https://www.youtube.com/watch?v=Blyb1D927pM&t=6597s):** And that, that may have been one of the smallest compute deals we've done. Um, yeah. **Dave Blundin — [1:50:01](https://www.youtube.com/watch?v=Blyb1D927pM&t=6601s):** Really? Wow. Are, are you actually leasing or buying or, or building or what are you doing? **Alexander Wissner-Gross — [1:50:07](https://www.youtube.com/watch?v=Blyb1D927pM&t=6607s):** Can't, can't comment. **Dave Blundin — [1:50:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=6608s):** Oh, a secret. Oh, sorry, I didn't know that. **Peter Diamandis — [1:50:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=6609s):** Welcome to the health section of Moonshots, brought to you by Fountain Life. You know, my mission is to help you use the latest technologies, including AI, to not just do your work at home, teach your kids, but to help you live a long and healthy life. I'm here today with an extraordinary physician, the chief medical officer of Fountain Life, Dr. Don Musalom. Don, let's talk about cancer. Uh, you know, I know, uh, from the member database that we've have at Fountain, our members who come in who think they're healthy, it turns out 3.3% of them have a cancer in their body they don't know about **Peter Diamandis — [1:51:08](https://www.youtube.com/watch?v=Blyb1D927pM&t=6668s):** Yeah, you know, it's interesting. People-- You don't feel the cancer until stage III or stage IV. And, and if you don't know what's going on inside your body, it's like driving your car with your eyes closed, and you can know. And so when members come through Fountain, how do they detect cancers? **Peter Diamandis — [1:51:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=6704s):** Yeah. So at the end of the day, you can know what's going on inside your body. It's your obligation to know. So check out Fountain Life. You can go to fountainlife.com/peter to get access to the latest technology to help you detect cancer at the very beginning, at stage I, when it is curable, before it gets to stage III or stage IV and you're a world of hurt. I'm gonna move us to our next story, which is about decision models. So here's the idea: when software needs to make a quick decision on a bounded decision, like is this a, you know, a transaction fraud, yes or no? You know, which of these five categories does a ticket belong in? It doesn't need a model that thinks for ten seconds and writes a paragraph. It needs an answer in milliseconds. That's called a decision model. Instead of generating, uh, text one token at a time, it answers a typed question or a choice, uh, a score with a yes or no quickly. So on September 15th, a startup called Typesafe AI came out after two years of stealth with something called Jev. Uh, it's a closed decision model they call a system one model. Uh, it's fast, intuitive kind of thinking as opposed to slow reasoning. So, uh, Alex, I'm gonna go to you, you know, and Salim. You guys were excited about Jev. Uh, let's parse it, understand what this is and why it's important. **Alexander Wissner-Gross — [1:53:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=6783s):** Yeah. So a bit of maybe prehistory first. In the beginning, there was the transformer, and the transformer was good, and it was based on an encoder and a decoder. So it, it took a sequence and converted the sequence to an embedding. That was the encoder part. And then it took the embedding, and it decoded that to another sequence or another part of a sequence. That's the decoder part. And the encoder part evolved into a whole ecosystem of models, popularly maybe BERT style models, and the decoder style or the decoder half of the transformer evolved into a much larger ecosystem of large language models. So most of the, the models that are consuming all of the compute of civilization today are decoder style transformers, and the encoder has basically gone missing in action. There are some people who still think maybe for purposes of retrieval, uh, it, it's still popular to, to, to do embedding based retrieval, but by and large, decoders have stolen the show. And, uh, fast-forwarding maybe year or two years ago at this point, OpenAI and others recognizing that there was a desire to be able to take decoder-only large language models and to add structure to them because without structure, you ask for next token, you can get anything. People wanted a little bit more structure like they were getting in, in the days of encoder-only models, where you get an embedding out and then you can train a classifier based on the embedding, and the classifier maybe has a finite set of categories or numerical outputs. OpenAI added structured output support for their GPT series, and everyone copied that. But no one, uh, a- as a fu-- my perception is, as a function of the total user base for all models, structured output never really got that much love, somewhat analogously to how OpenAI launched rec-- um, RFT, reinforcement fine-tuning, and no one used that, and they had to shut that down. Similarly, structured output was available, but I think underloved and underused by the community. So fast-forwarding then all the way to a few weeks ago, startup Typesafe announces a model called Jev, and Jev, they market as so-called system one intelligence. Th- this is Kahneman style reference to system one, system two type thinking. System one purportedly being intuitive, fast reaction. System two being reasoning, meditative, long-term type thinking models. And one could squint at Typesafe's announcement of Jev and say, "Okay, th- this is-- in some sense, this is a reaction to an overindexing or an overreaction, uh, by the, the user base of, of AI at this point to reasoning models, in some sense maybe arguing that we're using reasoning models too much, and we've abandoned the, the, the important use case of really fast, intuitive snap decision models." And this is purportedly then where Jev comes in. Jev purportedly, again, a whole new architecture. The, the detail's not quite clear that it can-- it, it won't output, uh, unless you, you torture it into it. Uh, it, it won't output general purpose sequences like, say, GPT or Claude will, but you can ask it sort of like a Magic 8-Ball. You, you can ask it to-- Is, is that too dated a reference? Maybe. **Peter Diamandis — [1:56:32](https://www.youtube.com/watch?v=Blyb1D927pM&t=6992s):** Nope. **Alexander Wissner-Gross — [1:56:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=6993s):** Nope. Uh, you, you, you can, you can feed it a sequence or an image or a multimodal input, uh, and you can ask it to, to answer in the form of a categorical, so it's one of N possibilities, or a numerical, like give me a number between zero and one on a continuum, or a binary, like true or false. You, you can ask for a much simpler output. And turns out that's just a great idea. Like, you can use it for, for use cases where, where you need really low latency snap decisions, like computer use assistance, like pressing buttons on a computer screen or for ultra low case classification problems if you want to classify every row in a database, decision style models, which are in some sense like a reinven- a reinvention of the encoder transformer type wheel. Amazing story. But then, of course, this is such a simple idea, you could ask the question: Why doesn't everyone else do this? And the answer is everyone else is now doing it. OpenAI released, as part of their dev day that we covered last time, a decisions API. So OpenAI immediately co-opted this renewed interest in so-called type one or decision models. This is already available via OpenAPI. There are open source projects that have all cloned this. Turns out this is like an absurdly, absurdly simple concept for everyone to just go and implement trivially. It could, in its simplest case, just be you take an, a Chinese open weight LLM off the shelf and you fine-tune it, uh, and maybe lobotomize it by a few levels just to produce categorical outputs on demand. That's the story. **Salim Ismail — [1:58:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=7083s):** So I th- I think this-- I think all of that is absolutely correct. The- there's a real, really important part of this for organizations, because when you're making decisions inside an organization, a large, vast number of those are system one type or, uh, thing. And I made a list of these just to make it clear for people. So, uh, route this ticket, um, approve this exception, uh, choose this supplier, escalate this transaction, um, move this thing left or right, send this message now or later. So when you're, when you have all these decisions to make, which are thousands of them, uh, you know, we've been working with these companies to do detailed task breakdowns, a vast majority of them are these little micro decisions. And trying to use an LLM for this is like trying to bring the Supreme Court together to decide what, uh, like, which checkout, which checkout line of the supermarket you should go to. And therefore, you've got this wonderful architecture now where you can route system one things to this very low cost, nearly free thing. And then for pondering deep important questions, strategy questions, where a hu- huge amount of human judgment type models are required, you can route it to those. So it's, it's a big deal for the organizational singularity world. Uh, we've been waiting for something like this, and Alex is correct. This has been possible for a long time, but now it's been made built into the systems. It's, it's made a very, uh, callable layer. And this is huge because the, the cost of micro coordination collapses, and that's really big. [[reminders/AI Economy/Cheap Decision Models Can Collapse the Cost of Organizational Coordination by Salim Ismail|∴]] **Peter Diamandis — [1:59:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=7174s):** Richard, your thoughts on this? **Richard Socher — [1:59:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=7177s):** Classifiers are back. Um, it makes a lot of sense. Um, you know, not every, uh, thing in, in software needs a very complex decoder, um, like Alex correctly said. Um, it's a really clever new way of making the old, which is classifiers, new by allowing a more general encoder and then quickly giving you classification results. I think it's, it's one of those ideas that is so beautiful. A lot of people thought, "Oh, why didn't we do that?" And didn't realize that could be so exciting for so many people. So now there are already various, like, Chinese open source versions of this that, uh, along various benchmarks are, are doing better. Um, and, and we'll likely see this come, and other large labs will likely follow suit. **Salim Ismail — [2:00:20](https://www.youtube.com/watch?v=Blyb1D927pM&t=7220s):** And, and just to reiterate, we've mentioned this already on the pod, but JEV stands for Jevons Paradox because the thesis here is we'll do now a massive amount more micro decision-making than we did before as a result. **Peter Diamandis — [2:00:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=7233s):** Nice. Dave, you wanna close us out here? **Dave Blundin — [2:00:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=7235s):** Yeah. It's a great case study, I think, in the tension between, uh, one mega model from Anthropic or OpenAI serving all humanity or open source and creativity and entrepreneurship building things you never would have thought of, but then you get the risk of cyber terrorism. You know, so that's the tension that we're with. But I, I've really wanted to build a box that you put at the side of the basketball court when you go to the Y, and it's got a little camera on it, costs next to nothing, and it's doing, like, all of the announcing that a professional announcer would do, it's doing while you're playing pickup at the YMCA. You could crank that out, like, in two seconds using a classifier that's really fast and snappy and funny. Uh, and but you need open source to, to build things like that. So super excited about the fact that we have open source still. Not sure- **Alexander Wissner-Gross — [2:01:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=7283s):** Actually, Dave- **Dave Blundin — [2:01:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=7284s):** ... not sure- **Alexander Wissner-Gross — [2:01:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=7284s):** Dave, you're making me think the Magic 8 Ball really should, whoever owns it, Mattel or whoever, they just, like, u- use- **Dave Blundin — [2:01:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=7290s):** Yeah **Alexander Wissner-Gross — [2:01:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=7290s):** ... a decision model to implement a modern Magic 8 Ball that actually understands the question and answers it with a categorical. **Dave Blundin — [2:01:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=7297s):** Totally. Totally would sell. It makes a ton of sense. Like 20 bucks. ## Project Meridian, Elon Musk, and the Future of Warfare **Peter Diamandis — [2:01:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=7301s):** Yeah. Or free on, free as an app. All right. I'm gonna close this out with a story from two days ago. So on Wednesday, the Defense Secretary Pete, uh, Segwith, uh, speaking at the Marine Corps base in Quantico, announced Project Meridian. It's a new Pentagon effort on the future of warfare. It's co-led by Elon Musk and Palmer Luckey, pretty extraordinary, along with Newt Gingrich, and overseen by the Pentagon's, uh, CTO, Emil Michael. Uh, let me read from Segwith, what Segwith said. He said, quote, "Project Meridian, the future of warfare, is not about developing new strategies or new policies. It's about discovering, developing, and fielding the weapons and systems future troops will need on the battlefield from the Earth to beyond the Moon." Uh, findings are due in 120 days. I, I like these kinds of commissions that are time limited and don't have Elon off on the side for, you know, a year at a time. It's worth noting in the context that, you know, both SpaceX and Anduril hold multi-billion dollar defense contracts, and Anduril is building autonomous weapons. So Richard, uh, your essay names autonomous weapons as one of the four genuine concerns, and you've said AI should never control lethal decisions without human oversight. Your thoughts on this? **Richard Socher — [2:02:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=7379s):** Yeah. Um, I, I stand by those. Um, it's not a, a particular area that I am excited about applying AI to. Um, obviously, like, people will. Um, but I really hope... I mean, you know, I, again, I'm not a doomer at all, but, like, a really poor decision would be to give AI access to all the nuclear codes and all the nuclear weapons and connect it. That's literally how Skynet in Terminator 3 gets started. So, uh, I think there are places for super intelligence and scientific discovery that I'm very excited about expanding human knowledge. I think, uh, the more, uh, we, we get to deciding not just to impact human lives but to end human lives, the more we, we should have human oversight. [[reminders/Deterrence/AI Must Not End Human Lives Without Human Oversight by Richard Socher|∴]] **Peter Diamandis — [2:03:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=7420s):** Mm-hmm. Yeah. Uh, Salim? **Salim Ismail — [2:03:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=7424s):** Um, you know, I think what's important about this whole thing is they're trying to revamp and rethink how you... this kind of 100-year-old organization. And the big question's gonna be, can a procurement organization built for 20-year weapons programs operate on, like, a 90-day technology cycle? And this is going to be the big challenge. Um, it's not that the risk-- It's not that they've failed to identify these future technologies, that we can all see those. It's how quickly can they absorb them and at the speed they're developing and bring them to the front. **Peter Diamandis — [2:04:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=7458s):** Yeah. **Salim Ismail — [2:04:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=7458s):** Like, you can't fight exponential technology with linear procurement. [[reminders/AI Superpower/Linear Procurement Cannot Fight Exponential Technology by Salim Ismail|∴]] **Peter Diamandis — [2:04:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=7463s):** Right. Uh, Dave, your thoughts please. **Dave Blundin — [2:04:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=7467s):** You know, I, I don't know. I, I, I had a great time with Palmer Luckey in LA, and I, I really, really love, uh, love him. And I think Elon too is just an awesome, good-natured person. I'm just overjoyed that there are people in Washington that I can sit down to-- with and relate to. Like, my entire career going to Washington has been a dread for me- **Peter Diamandis — [2:04:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=7486s):** Mm **Dave Blundin — [2:04:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=7486s):** ... because it's just lawyers and politicians and occasionally an accountant, and there's just no productive meeting. And for some reason, just in the last year, we're starting to see very, very smart, very capable people willing to go and, and get a mosquito bite, I guess- **Dave Blundin — [2:05:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=7502s):** ... in Washington. Um, and I, uh, just makes me really optimistic that they'll figure some things out. Um, I'm also not a fan of autonomous weapons that make decisions in the field, but Palmer Luckey made a very good case for it. You know, you can see it in our podcast from, uh, from LA. Uh, I, I don't agree that it's a good choice, but he a- he actually has some very rational arguments for why it's gonna be that way. **Peter Diamandis — [2:05:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=7526s):** Yeah. **Salim Ismail — [2:05:26](https://www.youtube.com/watch?v=Blyb1D927pM&t=7526s):** Can I, can I just mention one more statistic here? **Peter Diamandis — [2:05:28](https://www.youtube.com/watch?v=Blyb1D927pM&t=7528s):** Please. **Salim Ismail — [2:05:29](https://www.youtube.com/watch?v=Blyb1D927pM&t=7529s):** Uh, if you went back to the two years into the Ukraine-Russia conflict, they were using about a half a million drones to fight and prosecute the war. Um, this year Russia will make 10 million drones, and the Ukraine will make 10 million drones. So talk about exponential. Uh, that is an unbelievable ec- escalation, but without humans in the loop. So that's the good news around it. Uh, uh, of course, those drones are doing a ton more damage, but they're fighting each other with drones at a scalable level. The other statistic I remember, there's about 10,000 drones a month crossing the Mexico-US border. Uh, and so the problem there is wall technology is not as good as drone technology. And so trying to build a wall along there is not the greatest idea right now. **Peter Diamandis — [2:06:20](https://www.youtube.com/watch?v=Blyb1D927pM&t=7580s):** Nice. Alex, close us out on this one. **Alexander Wissner-Gross — [2:06:23](https://www.youtube.com/watch?v=Blyb1D927pM&t=7583s):** Yeah. A couple points. So the Secretary of War announced this alongside several other initiatives, maybe most conspicuously a s- a project codena- codenamed Project Agincourt, uh, which is the standup of the Department of War's first autonomous warfare command or Auto War Com. This, I think, is a transformative moment for the Department of War. We finally will have a dedicated joint force devoted to autonomous weapons systems, I think i- including drones, but n- not exclusively drones. Th- this is a major, major step forward for US capabilities to finally have a single joint force dedicated to this with a four-star functional combatant command that we've been missing. We're arguably missing... You know, I, I would go on a soapbox and say there-- I, if, if I were Secretary of War for a day, there are probably several other functional combatant commands that I'd spin up if I had the opportunity, but this would have been one of my top five list. The, the other point that I'll make just for Project Meridian, which you were asking about specifically, is in the announcement when, uh, SecWar Hegseth announced it, there was very particular language around examining, uh, I'll quote, "The full spectrum of future warfighting domains from subterranean depths to the cislunar frontier." So I, I think those two particular domains, the ocean bottom and cislunar and lunar space in general, are two wildly underserved domains, not just for warfare, but for peacetime activities as well. We know embarrassingly little about our ocean bottoms, and optimistically, one can imagine that if, if the DOW decides that it's now very interested in investing in American assets and American exploration and American dominance on the floor of the oceans and the lunar surface and the cislunar region in general, I think that is going to have dividends and pay dividends for the entire economy and for humanity's broader technological advances in a way that might naively have nothing to do with warfare or war fighting. **Peter Diamandis — [2:08:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=7714s):** Yeah. **Dave Blundin — [2:08:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=7714s):** That's one of the points that Palmer made actually in LA is that under the ocean it's not practical to communicate with a central server or central command, so that's already automated. And, uh, I don't know what triggers it, but once it goes into hunt mode, it just hunts, and it's not communicating back. **Peter Diamandis — [2:08:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=7731s):** Hm. **Alexander Wissner-Gross — [2:08:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=7731s):** Two-thirds of the Earth's surface, and we know embarrassingly little about what's hiding under it. **Peter Diamandis — [2:08:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=7734s):** Yeah, we know more, we know more about the surface of Mars than we do our ocean floor- ## AMA: AI, Jobs, Productivity, and the Attention Economy **Alexander Wissner-Gross — [2:08:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=7739s):** Yes **Peter Diamandis — [2:08:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=7739s):** ... uh, for sure. Uh, Richard, uh, this is the part where we answer our viewers' questions with an AMA. **Richard Socher — [2:09:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=7746s):** Yeah. **Peter Diamandis — [2:09:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=7746s):** And as our guest, I'm gonna give you first crack to choose one of these questions. So if you could, pick the number, read the question and who it's from, uh, and then dive in. **Richard Socher — [2:09:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=7758s):** All right. Well, uh, let me try to scan them really quick. Um, all right, "If AI makes companies 10X more productive but we don't need 10X the output, where does the value go? Shareholders, workers, or does it evaporate?" I think this is actually something I have thought about in the past. Uh, I think we can predict the impact of jobs, uh, in a certain industry from AI based on the elasticity of the demand when the price of that product goes massively down. We don't need to have billions and billions of illustrations in the world, and so when AI made the price of one illustration go down from 200 bucks to, like, two cents or less, like, we just didn't need as many illustrators anymore, um, uh, eh, because the demand for illustrations didn't go massively up. Yes, every little blog post and every little tweet can now have beautiful visualization and, and illustration, uh, but, uh, you know, we didn't need many more billions of them. Uh, I think software is a different one. You can actually have, everyone can have several pieces of software just specific to them. So we can actually have billions of different software products, uh, customized for each person, uh, and so the demand, uh, for that product will go up. Jevons paradox is gonna be alive, uh, in that world a- and we're going to see more and more demand, and there will be, uh, more, uh, value accruing to everyone. I think in terms of shareholders versus workers, I think the wave of AI, in the best scenario, will be a huge force for more entrepreneurship and, in the worst case scenario, a force of more inequality. I think, uh, everyone who owns some equity in a company that uses AI can love AI. [[reminders/Economic Transition/Falling Prices Decide Whether AI Creates Markets or Replaces Workers by Richard Socher|∴]] **Peter Diamandis — [2:11:03](https://www.youtube.com/watch?v=Blyb1D927pM&t=7863s):** Wonderful. Uh, Salim, over to you. **Salim Ismail — [2:11:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=7866s):** Uh, I will take, uh, question number three. Um, "If AI can write code, research, create marketing, manage projects, and outcomes, what is left for a college graduate in 2030?" And that is from JaredKuhn8812. So the obvious answer would be be emp- uh, have empathy and be c- uh, be creative, et cetera. But I think the bigger shift, which builds on what Richard just said, is you shift from doing tasks to focusing on owning outcomes, right? **Peter Diamandis — [2:11:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=7896s):** Mm-hmm. Yes. **Salim Ismail — [2:11:36](https://www.youtube.com/watch?v=Blyb1D927pM&t=7896s):** So a graduate used to be valuable because you could, you could execute research or build a spreadsheet or draft a deck, but now you wanna say, "Hey, here are the constraints, like, g-go figure out, and, and here's how we'll know whether we solved it." Or-- And then use an AI to sc- get to it, and it brings you back to what should your problem space be. And this is the massive opportunity because we traditionally learn judgment by doing the groundwork. Uh, we need a totally new ap- apprenticeship model, uh, when the groundwork disappears, and this is a huge challenge for the education system. But the change is going to be focusing on what problems you wanna solve and then letting, orchestrating the g- forces that will help you solve that problem. [[reminders/Entrepreneurship/Graduates Must Move From Completing Tasks to Owning Outcomes by Salim Ismail|∴]] **Peter Diamandis — [2:12:19](https://www.youtube.com/watch?v=Blyb1D927pM&t=7939s):** Yeah, Jared, find your purpose, right? A passion is something you love doing. A purpose is something you love doing that helps other people. Uh, build- **Salim Ismail — [2:12:25](https://www.youtube.com/watch?v=Blyb1D927pM&t=7945s):** Make sure it's massive and it's transformative- **Peter Diamandis — [2:12:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=7947s):** Yeah **Salim Ismail — [2:12:27](https://www.youtube.com/watch?v=Blyb1D927pM&t=7947s):** ... and it's purposeful. **Peter Diamandis — [2:12:28](https://www.youtube.com/watch?v=Blyb1D927pM&t=7948s):** And, and build a company, and then direct AI to implement it. It is your workforce. Uh, Dave, over to you. **Dave Blundin — [2:12:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=7958s):** Uh, I can't resist number four. I love all these questions so I'm, I'm torn. But, "How can data centers make neighborhoods richer instead of the owners?" And that's from LMBatman66. Um, so, uh, I took a tour of the Lowell Data Center, Jeff Markley, Markley Data Center, and that thing is creating wealth in that neighborhood like you wouldn't believe. And, uh, the way it works fundamentally is the data center is so immensely valuable, and the town budget is maybe a couple million dollars a year. So between the tax revenue, the donations, and the job creation, the town is thriving, and it, and it's a town that really needed it too. So I think it's happening very naturally. Uh, what you wanna do is attract a data center to your neighborhood first and foremost, and then you have the next five or seven years to figure out your tax policy, your donation policy. I tell you, these data center operators are very interested in great PR, and so they, they will donate like crazy to the high schools, to the neighborhoods, to... They... It's really, really working. **Peter Diamandis — [2:13:34](https://www.youtube.com/watch?v=Blyb1D927pM&t=8014s):** Yeah. We're gonna see a whole-- We're gonna see an entire shift where data centers are offering such benefits on jobs, on tax breaks, on lower cost energy, uh, that you're gonna be begging to have a data center in your backyard. Alex, number two's for you. **Richard Socher — [2:13:48](https://www.youtube.com/watch?v=Blyb1D927pM&t=8028s):** All right, so number two asks, "If every major tech platform started open and democratic, then consolidated power, Google, Meta, Amazon, why would AI be different?" And this is from OpenSourceMind. The premise of the question is half right, half wrong. I, I, I'm not sure I buy the premise that consolidated power, w- the subtext of which, uh, in... One can juxtapose it with the, the user handle OpenSourceMind. I, I'm not sure the framing is the right framing. I, I would agree that in every major tech revolution, initially the barrier to entry is low because there's some new platform innovation and then you see lots and lots of players enter the field. And then as the field matures, you see economies of scale and a deeper bench of infrastructure typically supporting it, and as a result, that favors larger and larger players, and you do see consolidation. But the, the subtext of the question that it's somehow anti-democratic or not open, I don't agree with that premise in the least. I, I do think as you see consolidation in an industry, I think it's important to be vigilant from an antitrust perspective to make sure that it remains competitive, but the premise that hyperscaling is somehow n- closed or anti-democratic, don't buy the premise at all. **Peter Diamandis — [2:15:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=8113s):** Hmm. All right. Uh, Richard, as our guest, you get first crack once again. Take a look **Richard Socher — [2:15:24](https://www.youtube.com/watch?v=Blyb1D927pM&t=8124s):** I do think the internet ad-based economy, uh, will be under pressure. Where- **Peter Diamandis — [2:15:29](https://www.youtube.com/watch?v=Blyb1D927pM&t=8129s):** Which, which, which question are you answering? **Richard Socher — [2:15:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=8130s):** So first question, if AI agents outnumber humans from one to two years on the internet, doesn't the, uh, entire ad-based economy collapse? Uh, what repl- replaces the attention economy? I think there, there's actually a really deep answer here. I'll try to summarize it, but like one, we already have more bots on the internet, uh, and more agents on the internet than people. So this-- I predicted this, uh, last year, and it happened a few months ago. Um, so that's number one. I do think we're seeing the first kind of, um, sort of skirmishes, uh, in that, um, when, for instance, uh, various agents try to make purchases on Amazon without really being on the Amazon platform, and Amazon usually tries to turn them off 'cause they wanna own that relationship directly with the customer, understandably. Um, but it's just convenient for someone to just say, "Just go buy these batteries," and they don't care which batteries it is. And so if you have that control over which one it is, you can start selling that, uh, to, uh, other, uh, sellers and, and, uh, people who create physical goods. And so there is, uh, going to be, uh, continued-- uh, like ads will continue to be important, and in fact, if you think about a fully abundant society, the one thing that you cannot scale exponentially is the hours in the day that people can pay attention to you, that can make you famous. Uh, and so fame and brand and network effects, uh, and other things like that will become more and more of a currency. And so the attention economy, which is connected slightly, uh, not necessarily, uh, exactly equivalent to the ad-based economy, uh, that actually will become a bigger thing as we have more and more of our material needs met by, uh, technology. **Peter Diamandis — [2:17:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=8235s):** Great. Uh, Salim. **Salim Ismail — [2:17:18](https://www.youtube.com/watch?v=Blyb1D927pM&t=8238s):** I will take number, uh, seven. If we remove 10X the cars from the street, what happens to the insurance industry? We won't need driver's insurance, question mark? And that's from Robert Zerby JB10H. Um, so, uh, you know, we talked about how, um, uh, liability lawyers won't be needed, and I got a huge flame from a bunch of folks saying, "Hey, we really protect the citizenry." And I, I... There are. Just-- So I'm just apologizing because there's a spectrum of people, and some people are full ambulance chasers, and other people do things. There's a whole segment of this called ethical lawsuits where people get together and try and, uh, sue big companies for doing the right ethical thing. And so that's an important, uh, segment of that. So I just wanna acknowledge that side of it. But just to answer the question, um, you know, the, the, the thing is, industries don't, uh, disappear when the risk changes. You re- change the risk, right? So driver liability may fail, software liability rises, right? You've-- You-- What's your, uh, cyber risk? What's your manufacturer liability risk? Um, so the insu- the insurance transitions from did, did Salim crash, to which layer of the autonomous stack failed? And, and we, we've had this before, and in product liability, it's different layers. So we'll end up with the same type of model. You shift the insurance risk to a different level. **Peter Diamandis — [2:18:41](https://www.youtube.com/watch?v=Blyb1D927pM&t=8321s):** And there's gonna be all kinds of new insurance markets for humanoid robots- **Salim Ismail — [2:18:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=8324s):** Mm-hmm **Peter Diamandis — [2:18:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=8324s):** ... for flying cars, for drones, for all kinds of things. **Dave Blundin — [2:18:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=8326s):** Well, just a data point on that, you know, a single big data center like Abilene, Texas, is half a trillion dollars. All the cars combined are four trillion. One data center is half a trillion, and it's in a tornado alley. You think you probably wanna insure that. So the things-- number of things that need insurance is going up 10X just with the economy going up 10X. **Salim Ismail — [2:19:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=8345s):** Dave- **Dave Blundin — [2:19:05](https://www.youtube.com/watch?v=Blyb1D927pM&t=8345s):** It just needs to move **Salim Ismail — [2:19:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=8346s):** ... if you really, if you really wanna be brave, you can't get home insurance in Florida anymore. **Peter Diamandis — [2:19:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=8350s):** Yeah. **Salim Ismail — [2:19:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=8350s):** So go create an insurance company for that. **Peter Diamandis — [2:19:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=8353s):** Nice. Uh, Dave, pick your question. **Dave Blundin — [2:19:17](https://www.youtube.com/watch?v=Blyb1D927pM&t=8357s):** Uh, I like number eight. Uh, what is the lowest possible job in an AI civilization? What an, what an interesting question. I-- Uh, that's from LBNODK. Um, yeah, you know, I, I saw this pile of a million valves for a liquid-cooled data center. A million freaking valves, and I'm like, how do those actually end up in pipes? The humanoid robots that can install those things are pretty far out. It's a very subtle process to, to install those, so that job will be around for a long time. And then they're paying a lot for it. But that's not the lowest. But I'm tellin' you, it's r- it's hard to think, like, what is the lowest surviving job? Do you guys have any thoughts? **Richard Socher — [2:19:54](https://www.youtube.com/watch?v=Blyb1D927pM&t=8394s):** S- so many thoughts. If, if I may, uh, it-- I, I wanna construe the question as in a pure AI civilization, in which case, arguably, the way that you measure low is the, the job that requires the least compute, in which case the, the jobs that require the least compute as a result are the least economically valuable if, if their inputs are the lowest, might ironically look like the most valuable jobs in a pre-AI civilization, more of that paradox style. So the, the, the great writers, businesspeople, uh, the, the, the creative actors that Ayn Rand may fetishize would ironically in a post-AI civilization be the lowest possible jobs because more of that paradox, paradox style, those were the first ones to be automated. **Peter Diamandis — [2:20:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=8440s):** Hmm. Interesting. All right, Richard. **Salim Ismail — [2:20:42](https://www.youtube.com/watch?v=Blyb1D927pM&t=8442s):** I think there's two polarities here. **Richard Socher — [2:20:43](https://www.youtube.com/watch?v=Blyb1D927pM&t=8443s):** I do think- **Salim Ismail — [2:20:43](https://www.youtube.com/watch?v=Blyb1D927pM&t=8443s):** One is very high judgment, per what you were just saying, Alex, and second is very physical, highly contextual work. **Richard Socher — [2:20:51](https://www.youtube.com/watch?v=Blyb1D927pM&t=8451s):** I would, I would argue maybe also low is just in terms of how valuable, uh, and, and, and moral society deems those kinds of jobs. And I do think actually that, uh, the other question about the cars on the street, um, I don't think we removed, uh, 10X the cars when we have self-driving. But self-driving might be making things so much safer that indeed, uh, people don't need as much accident insurance and no-- not as many ER people, not as many ambulances. And so in a weird way, you make objectively the world better by reducing traffic deaths. But it does actually have potentially a mildly negative effect on parts of the economy, and we should all be rooting for that, uh, in this case, right? And so I think the lowest jobs in terms of moral standing are things where you don't really progress, uh, humanity forward. And my hunch is they're the types of jobs in entertainment, um, that, that people will value a lot, and, and fame and attention, uh, will become, uh, more of a currency, uh, in that world. And so if your job doesn't get you any of that, it might be considered, uh, lower in, in that future. **Dave Blundin — [2:21:56](https://www.youtube.com/watch?v=Blyb1D927pM&t=8516s):** I think the lowest job that just will never go away will be something in politics, where it's completely irrelevant already, but but it's just there, and it'll stay there, and no one's gonna change it. **Peter Diamandis — [2:22:06](https://www.youtube.com/watch?v=Blyb1D927pM&t=8526s):** Nice. All right, Alex- **Salim Ismail — [2:22:07](https://www.youtube.com/watch?v=Blyb1D927pM&t=8527s):** Imad, I'll... I- I gotta throw in Imad's perspective. He always thinks it's the BART train driver because it's unionized to hell. **Dave Blundin — [2:22:13](https://www.youtube.com/watch?v=Blyb1D927pM&t=8533s):** Yeah, yeah. There you go. **Peter Diamandis — [2:22:15](https://www.youtube.com/watch?v=Blyb1D927pM&t=8535s):** Alex, number six. Close us out. **Alexander Wissner-Gross — [2:22:16](https://www.youtube.com/watch?v=Blyb1D927pM&t=8536s):** Number six, "If we cure illnesses that are often due to bad behavior, what's going to take care of the cause?" Joe Wilder. I assume the cause is reference to people choosing to behave, quote-unquote, badly. And the answer, I... This is another case of my not buying the premise. If you look at some of the really spectacular results that have been coming out of GLP-1 class studies, including third and soon presumably fourth generation GLP-1 RAs, they're actually addressing the cause. Like, addictive behaviors are being mitigated or at least partially treated by the same drugs that are curing in, or at least treating, I have to, to caveat that, inflammation and, uh, blood sugar, diabetes, all of these other conditions. The, the root causes, which is... or human behavior are themselves, the, the human behavior is being affected by the same drugs. So I, I- **Dave Blundin — [2:23:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=8590s):** I'm so optimistic about what Alex is saying- **Alexander Wissner-Gross — [2:23:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=8592s):** Yeah **Dave Blundin — [2:23:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=8592s):** ... 'cause I really fully believe AI, this is gonna be one of the highest callings of AI very, very soon, is to make you feel really good about doing good things and happy as you're doing it. **Peter Diamandis — [2:23:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=8602s):** Mm-hmm. Love that. **Alexander Wissner-Gross — [2:23:22](https://www.youtube.com/watch?v=Blyb1D927pM&t=8602s):** And not wa- not want to do bad things for yourself. It turns out we have the capability now. We figured out how to do that, at least in part, and we're gonna figure out a lot more. ## Closing Thoughts and The Eureka Machine **Peter Diamandis — [2:23:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=8610s):** All right. As always, a call out to our amazing community. If you've got a music video that you'd like to show as outro, please send it to the team at [email protected]. And speaking about amazing outros, here is Abundance by Stephen Gross. Uh, my dear moonshot mates, uh, this is the real you, so check it out. **Peter Diamandis — [2:24:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=8673s):** Drink, Alex. Drink. **Alexander Wissner-Gross — [2:24:35](https://www.youtube.com/watch?v=Blyb1D927pM&t=8675s):** Drink water. **Dave Blundin — [2:24:37](https://www.youtube.com/watch?v=Blyb1D927pM&t=8677s):** Martini glass. **Peter Diamandis — [2:24:38](https://www.youtube.com/watch?v=Blyb1D927pM&t=8678s):** Oh, that was fun. That was- **Salim Ismail — [2:24:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=8680s):** Friday night **Peter Diamandis — [2:24:40](https://www.youtube.com/watch?v=Blyb1D927pM&t=8680s):** ... that was Moonshots Live 2040 on the moon. Nice. Uh, so- **Alexander Wissner-Gross — [2:24:44](https://www.youtube.com/watch?v=Blyb1D927pM&t=8684s):** I don't think we should have to wait that long. **Peter Diamandis — [2:24:46](https://www.youtube.com/watch?v=Blyb1D927pM&t=8686s):** Yeah. Uh, Richard, your new book, The Eureka Machine, wherever you purchase your books. Do you have the Audible out? **Richard Socher — [2:24:53](https://www.youtube.com/watch?v=Blyb1D927pM&t=8693s):** Uh, it should come out very soon. Yeah. **Peter Diamandis — [2:24:55](https://www.youtube.com/watch?v=Blyb1D927pM&t=8695s):** All right. I'm an Audible reader- **Richard Socher — [2:24:57](https://www.youtube.com/watch?v=Blyb1D927pM&t=8697s):** I- **Peter Diamandis — [2:24:57](https://www.youtube.com/watch?v=Blyb1D927pM&t=8697s):** ... but I have skimmed the book here. Uh, congratulations on this. **Dave Blundin — [2:24:59](https://www.youtube.com/watch?v=Blyb1D927pM&t=8699s):** I gotta say, Richard, there's... I gotta say something. **Peter Diamandis — [2:25:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=8702s):** Please. **Dave Blundin — [2:25:02](https://www.youtube.com/watch?v=Blyb1D927pM&t=8702s):** What you're doing with the Recursive looks like such an incredible opportunity to move humanity forward, so congrats on taking that on. **Richard Socher — [2:25:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=8709s):** Yeah, so cool. **Peter Diamandis — [2:25:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=8709s):** Thank you. **Richard Socher — [2:25:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=8709s):** Thank you. **Dave Blundin — [2:25:09](https://www.youtube.com/watch?v=Blyb1D927pM&t=8709s):** Amazing maps. **Richard Socher — [2:25:10](https://www.youtube.com/watch?v=Blyb1D927pM&t=8710s):** We'll try to make it a good one for humanity. **Alexander Wissner-Gross — [2:25:12](https://www.youtube.com/watch?v=Blyb1D927pM&t=8712s):** And, and Richard, please resist the urge to have you.com acquire your own frontier lab like everyone else seems to be using, spinning off their own frontier labs, not to name names, uh, a- as a, a financial engineering exercise to maximize their equity in their original startup. Please resist the urge to, to follow that trend. **Richard Socher — [2:25:30](https://www.youtube.com/watch?v=Blyb1D927pM&t=8730s):** All right. **Peter Diamandis — [2:25:31](https://www.youtube.com/watch?v=Blyb1D927pM&t=8731s):** All right. I love you guys. **Richard Socher — [2:25:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=8733s):** Thank you so much. **Peter Diamandis — [2:25:33](https://www.youtube.com/watch?v=Blyb1D927pM&t=8733s):** Be well. Until next time. --- ## Conversion handoff The final research edition should convert this complete source into independently viable quotation objects arranged in transcript order. It should preserve larger contiguous thoughts, exclude sponsor material and empty banter, restore missing subjects with square brackets, use ellipses only for genuine omissions or interjections, label substantial synthesis as **Adapted from**, and retain linked timestamps. Claims should remain attributed claims until independently verified. Later Reminder production must return to the recording at the linked timecode and must not treat this prepared transcript as independent factual verification. <!-- BEGIN ASI RSI TIMELINE ONTOLOGY 2026-10-03 --> ## Developed Wiki Ontology This source dossier now has a reciprocal exploration layer in [[ASI and RSI Timeline Ontology]]. Canonical wiki nodes preserve concepts, organizations, programs, people, and software as durable objects before any quotation is promoted into a Reminder. - [[ASI and RSI Timeline Ontology#Scientific Intelligence and Recursive Improvement|Scientific Intelligence and Recursive Improvement]] — Scientific discovery loops, recursive improvement, open-ended search, and machine-generated scientific formalisms. - [[ASI and RSI Timeline Ontology#Computational Biology and Virtual Cells|Computational Biology and Virtual Cells]] — Virtual cells, organoids, biological foundation models, programmable organisms, and the model-to-laboratory loop. - [[ASI and RSI Timeline Ontology#Neural Decoding and Cognitive Systems|Neural Decoding and Cognitive Systems]] — Brain imaging, neural encoding and decoding, cognition, biological algorithms, and high-bandwidth brain-computer interfaces. - [[ASI and RSI Timeline Ontology#Generative Humans, Continuity, and Presence|Generative Humans, Continuity, and Presence]] — Interactive avatars, synthetic presence, preserved biological materials, personality reconstruction, and identity continuity. - [[ASI and RSI Timeline Ontology#Model Architecture and Compute Economics|Model Architecture and Compute Economics]] — Compact models, knowledge separation, orchestration, decision systems, compute ownership, and price-performance frontiers. - [[ASI and RSI Timeline Ontology#AI Control, Law, and Alignment|AI Control, Law, and Alignment]] — Containment, shutdown, liability, behavioral constraints, constitutional design, and the enforceability of model regulation. - [[ASI and RSI Timeline Ontology#Physical Intelligence and Scientific Infrastructure|Physical Intelligence and Scientific Infrastructure]] — Semiconductors, machinery, fusion, materials, space infrastructure, and the computational substrates of physical capability. - [[ASI and RSI Timeline Ontology#Autonomous Warfare and Strategic Systems|Autonomous Warfare and Strategic Systems]] — Autonomous military systems, defense acquisition, undersea and cislunar domains, and institutional adaptation to exponential technology. Existing canonical pages are used instead of duplicate aliases for [[Functional Magnetic Resonance Imaging|fMRI]], [[Cellular Digital Twins|digital twins of cells]], [[Digital Twin]], [[Superintelligence|artificial superintelligence]], [[Open-Ended Evolution|open-ended algorithms]], and [[neural foundation models|neural foundation models]]. <!-- END ASI RSI TIMELINE ONTOLOGY 2026-10-03 -->