0:00Bernie Sanders sent a formal letter to the CEOs of Anthropic, Meta, and OpenAI.
0:04AI capabilities have reached a critical threshold. Pause AI development.
0:11The cat's out of the bag. It's too late, right?
0:14Nvidia just announced [music] a partnership that redefines what GPU compute means as a financial asset.
0:20This is like the very first pitch of the first inning of the build out of the Dyson swarm.
0:24Financial assets want predictable depreciation, and exponential technologies don't give you predictable depreciation.
0:31I'm not as concerned that this will end up being another mortgage-backed securities fiasco for a few reasons. One is Elon just dropped Grok 4.6, and he's right, it's a banger. Grok 4.5 came out 2 weeks ago, 4.6 just this morning, and 4.7 is rumored to be coming out in 2 weeks.
0:504.7, and this is what I think is going to be really interesting, is going to be trained on all the SpaceX physics and engineering knowledge. You can't get beyond frontier with this stuff, but what if you have the best engineers?
1:03Now, that's a moonshot, ladies and gentlemen.
1:08Welcome back to Moonshots, everybody. Uh we're going to be covering nine stories today that span the frontier from longevity, AI infrastructure, synthetic biology, AI filmmaking, urban air mobility, and oh, by the way, Grok 4.6 is crushing it. Uh the through line is the same as always. You know, accelerating singularity is compressing the distance between the impossible and the inevitable. We're back this week with the moonshot quintet.
1:35Uh AWG, our in-house superintelligence, Alex, welcome. Good to see you in your normal haunt.
1:41Good to be superintelligent.
1:42You are, my friend. Dave Blundin, our investor in AI extraordinaire.
1:48I was over at MIT all day today, hence the garb, but uh our Tech Trek teams just got back from SF, and they brought back huge amounts of knowledge, and they're all over at MIT CSAIL dispersing it across the campus right now.
2:00Love it. And Salim Ismail, our impresario of exponential organizations, Salim, where are you today? Still in Toronto?
2:09I'm still in Toronto heading back tomorrow.
2:13I should be going back now, but this podcast happened and I can't fly, you know, within like 6 hours of Yeah, [laughter] yeah, we're going to we're going to put a Starlink on your on your hat and have you walk around with it.
2:25Uh and back by popular demand, Emad Mostaque, the embodiment of the intelligent internet. Emad, I hope you're reading the comments on the last pod uh that we did together. People are loving you. Have you seen them?
2:39Uh yeah, I did see some of them. I don't normally read the comments, so I made an exception and it's very kind of everyone said.
2:45Yeah, it's the British accent, that's all it is.
2:47Yeah, it is. It is. It works for my American wife, it works for the others, you know.
2:52[laughter] Um Peter Diamandis, your host, your optimism amplifier. Please remember that having an optimistic and abundance mindset is a choice and our mission here is to deliver data-driven optimism that helps you make that choice. So guys, uh last night I was in Utah at the University of Utah. It was the finals for the $101 million longevity or shall I say health span X prize. Super psyched. We awarded a
3:20million dollars to 10 teams, recognized another 10 finalists. This is on the march to winning the $80 million grand prize. We've given away 20 million so far. And the mission of these teams, and you know, everybody please get excited about this, is to add 20 healthy years on your life, asking teams to reverse your functional losses that we get through aging in cognition, give you the ability to think and have memory like you did 20 years ago, in muscle, the
3:48ability to build muscle like you did 20 years ago, and in your immune system. Uh and it's extraordinary. We had 800 teams enter this competition. Every possible approach from mitochondria to uh you know stem cells to you know gene editing. It's it's extraordinary. So, this is a competition that will be won by 2030. So, keep an eye on this. Uh you know, Salim, I don't know if you want to jump in on this one.
4:20Your 2020 is infinity, right?
4:22Yeah. Well, you know, I think there's there's the X Prize was such a huge inspiration when I was writing the EXO book because you're reaching outside and I mean 800 teams, is that I think that's a record for the traditional prizes for us, right? It's kind of an incredible thing.
4:43Elon's $100 million prize for carbon removal ended up with something like 1,500 teams. But, you know, reversing aging, it's got to be tougher. And by the way, the teams here don't do this in theory. They don't do this in mice. They have to actually do human trials. So, they're all going to be doing trials with control groups and and humans probably around 150 people in the trial.
5:09So, it's real data and we're going to actually know which of these 20 to 30 approaches that actually make it to the finals work. Uh and for me, you know, Alex, we were talking about being in the midst of longevity escape velocity, this is accelerating it.
5:25It's going to be spiky, I think. I I still think just as with AGI either in our rearview mirror, as I think, or some might say it's either here or almost here. I I do still expect that health span and longevity escape velocity, it's going to be spiky and and I'm just optimistic that we can even out the spikes even if there's a subpopulation that achieves health span on the margin a few years before everyone else.
5:51You know, one of the most important things I think about this competition, when we launched the $10 million Ansari X Prize for spaceflight back in 1996, you know, back then people did not believe in commercial spaceflight. They didn't believe that individual teams could do this and carry humans compared to the government. And as it progressed, the confidence level in this, and then when it was won, money flowed in, regulations changed, you know, uh Bezos and Musk started, uh you know, Blue
6:20Origin and SpaceX, and it be changed the game. So, I'm feeling the same thing going on right now. You know, longevity, the idea of reversing aging has been sort of a a crackpot idea for most of the last few decades while I've been in industry, and it's beginning to change. People are starting to believe, yes, it's going to happen. Yes, we're in this health span revolution.
6:43We're getting to a We're getting to a when, not an if, question, right? And that's really huge. I I want to point out something that was that's really important, you know, Peter. When you launched the Ansari X Prize, there was no space industry to speak of, at least in the commercial side. And now we have a trillion-dollar industry that did not exist, right? Because the collective innovation plus all the members of all the teams end up going to SpaceX, Blue Origin, etc., etc. And we could expect the same thing to happen here, where you end up with a essentially you're creating a hundred
7:13and one million-dollar bet. You're building a portfolio of experiments and allocating capital to wherever there's demonstrated results. It's an unbelievable model that we've now seen repeat over and over again. It's incredibly exciting to see.
7:27Yeah, I was going to say Pete Peter, on this one. I I always ask the counterfactual question. Is Is there something having now run this health span prize, or at least the the beginnings of it, do you think that if this prize had been created, say, 20 or 30 years ago, that we could have made on margin enormous progress, or do you think there's some historic contingency that means right now is really the the first time in history where we could make credible progress on it?
7:55Yeah, I I think we had a lot of comments. We had a lot of the top scientists, Aubrey de Grey was there last night who coined the term longevity escape velocity, and then and then Ray popularized it. Uh I think everybody was at the consensus that the timing is perfect. That the tools for genome sequencing, for uh you know, making specific molecules, for being able to measure uh and and report with AI uh are the tools that are required today.
8:23I mean, there might have been some approaches uh that could have been done 20 years ago, but I think today uh is when we're going to make the greatest progress.
8:33Uh and I'm starting to see, you know, capital flowing in uh aggressively. At the end of the day, longevity is going to be the biggest market. Right? If you could add 30, you know, I've had this conversation, you know, Celine probably uh you have as well on stage, you know, with YPO audiences or family offices, and I say, "How much of your wealth would you spend for an extra 30 years of life?"
8:55The honest answer is nearly all of it.
8:59Yeah, it's I mean, you and and the powerful distinction there's not so much lifespan, but the healthspan effects are really, really powerful.
9:08Yeah, the numbers today are for the United States, if you if you look at it, uh basically uh 100 years ago in 1900, the average life expectancy was 47. Today, it's 79. We added about 2 months per year over the last century.
9:27And today, while the lifespan is 79, you're healthy until average age 63, and you spend the last 19 years of your 16 years of your life in poor health. And so when I was with talking to Crocs about this, I said listen, the biggest benefit the US budget could have and US economy could have is add 20 healthy years to people's lives. They're they're retiring later.
9:53They're not spending as much government money on on, you know, sick care. So it could be a huge transformation. Alex, do you want to say anything?
10:02Yeah, I maybe just again coming back to this historic counterfactual. I one of the things that irks me the most is that so many of the abundance oriented futures that we want to find ourselves in like LEV, the super intelligence solve everything just take forever and I I do wonder again where is it 20 years post Yamanaka that Yamanaka I think was 2006.
10:28We're in 2026. What did we blow these 20 years on? Why couldn't we have done this 10, 20, 30 years ago?
10:36I got a question for you on that. I'll answer that. I'll answer that.
10:40I think when you get multiple exponential technologies that can address this particular space, then that's the point to invest or put up a prize because then you get radical outcomes at very low cost. And so maybe there's a benchmark of the minute some domain has two or three or more exponential technologies converging on it, that's the point to have put dollars into something like this.
11:05[clears throat] Dave?
11:06Well, there's no doubt in my mind that I'm going to do more productive work in the second half of 2026 than in my entire life combined up till 2026. So throw it back at you Alex like even if we had worked really hard on this 20 years ago, uh would anything that we did between then and today even hold a candle to what we'll achieve between now and the end of the year cuz I mean it's so funny to me you know my my daughter is over at Moderna. She's a biochemical engineer.
11:32And she listens to the pod, of course, you know. And it's so obvious to her that we're in an AGI hard takeoff now, and the people she works with in biotech are about 1% AI aware and 99% not aware, but the AI aware now people are spending over half their day, maybe 80, 90% of their day talking to AI agents, and not in meetings, and not, you know, running gels, and not running assays, because it's it's just a different mode of living that's accelerating
12:02tremendously. But the the aware subset is tiny.
12:06I I almost think I I think it's an important point, and I almost think we need a new term for it. I I I just thinking off the cuff, maybe like retrospective hyper deflation, this very singularity oriented idea that with superintelligence, you discover that everything that you spent the past decades on was just a total waste, and you should have instead just done nothing, twiddled your thumb for decades, waited for superintelligence to solve it for you.
12:31Or or just work on, you know, chip fabs or something that will, you know, they'll be very useful on that day. It's like buying land in Texas or something.
12:38All these people, like who who who spent six-year PhDs trying to divine protein structures just mostly wasted.
12:46Well, that one was was really outed by Demis. Yeah, I mean, that one it What it was like an average of 4 years to discover one fold.
12:54I I knew there was a reason I didn't do a graduate degree. That was it.
12:58Cuz [laughter] it would have been irrelevant.
12:59That's a tough justification, Selena.
13:01That's right. What was the justification in all this?
13:04I mean, it could it could be worse. You could be a pure mathematician, right?
13:07Like [laughter] They're all good, too. They're They're They're having this this same moment of ennui.
13:13Yeah, but at least with the biologists, the longevity people, you can do assays and things physically. No, look, I think that this is the biggest market in the world, as you said, but it's the first time that it's tractable. I I if you go back to when there were the Yamanaka factors, all these other things, you didn't have the infrastructure necessary and the talent pool necessary.
13:32[snorts] I think that as you've seen the various breakthroughs and other things and everything come together, right now there will be a shortage of people that can really work on longevity properly even though it is the biggest market in the world. And 10 years ago, 20 years ago, that would have been even tinier.
13:48There were only a few people actually looking at some of these things back at that time. So, I think it is this confluence factor all coming together. And like I said, it's finally tractable. So, the people who are winning in the longevity X Prize, any indication that those things will work and I don't think they'll be short of capital. But again, the X Prize is the catalyst, right? And that was their whole idea.
14:10You know, Peter, you've pointed out something that I think is incredibly important to highlight, right? Which is that we spend today huge amounts of money on treating chronic diseases at the end of life.
14:20is Salim, it's uh the global cost of age-related disease is 20 trillion dollars per year.
14:27Yeah, it's a huge amount of Daniel Kraft just like 85% of health care costs are the last 5 years of your life. Type thing. It's a staggering number, right? Because now the business globe the global economy is 120, 130 trillion.
14:4120 trillion of that Look at that number, right? So, now now the business model becomes how do you maintain decent function, bodily function, before the disease appears and you'll completely change health care economics.
14:55And and global economics in general.
14:57In terms of productivity and wasted capital.
15:00Well, I think this is where this is where the peptides have come really interesting, right? Like it was like to lose weight, you had to work out. And that stopped, you know? You can get rid of that extra weight. Yeah.
15:09I'm going to report in on the pod.
15:10I'm going to get it reported in, see how my weight loss goes. But the thing is Which peptides am I taking? Do tell.
15:16I was going to try the retatrutide one.
15:24But my I know I'm going to lose weight, right?
15:26How are you getting access to No, but seriously, okay. How are you getting access to even Oh, you're in Canada?
15:32No, no, I'm not in Canada. I get it in the US. It's totally possible.
15:35know, there are drug dealers out there.
15:37Black [laughter] market drug dealers.
15:38Peptide dealers, right there you go.
15:40I I I guess Salim, that'll just be our little secret. There's no one paying attention to us. Nobody's Nobody's watching. Nobody's listening.
15:45But but this is how like you know that you're going to lose weight with it, right? And that's the first. And so now the concept of you can take a pill or an intervention and you could live longer. Well, you're already losing weight like that. And so I think again, that's a big awareness that's caused the longevity market to get even bigger.
16:02Yeah, just right. Your GLP-1s are the first longevity drug, uh, you know, in a lot of people's opinions. And it's one of the biggest grossing drugs, if not the largest one in human history.
16:13I I've gone even further that than that just again, bit of back-of-the-envelope calculations suggesting, and again, this is not medical advice, uh, that GLP-1s, especially third and maybe fourth generation GLP-1s, may actually be, when I refer to LEV, longevity escape velocity, as being potentially spiky, a pretty big spike.
16:34There There have been studies done recently on uh, some of the third generation, if memory serves, GLP-1s that suggest, uh, and again, not medical advice, that we may be, uh, or at least some subpopulation of humans that have undergone GLP-1 studies, maybe at something like 70% LEV just with GLP-1 therapy. So, if if that is the case, and again, encourage folks to to do their own, uh, independent back-of-the-envelope analysis, that's a heck of a an LEV spike in a
17:04subpopulation that's being administered GLP-1s. Yeah. Anyway, watch this space, everybody. It's exciting.
17:11Um, as you looked at the finalists, cuz I couldn't make it to the finals, but have you been tracking some of the teams, etc.? You you had a great snapshot view. Are you in the view that LEV we get to it by 2030 or earlier?
17:25So, here's the idea, right? If any of these teams win and can reverse your functional age, right? This is not a number from a you know a particular blood test you take, you know, that changes on the back of a form. This is are you feeling functionally younger? Are you know, do you have better muscle building capability, better memory, better immune system? So, what really matters is your function. And so, if we can do that, if
17:52we can actually reverse the clock by 20 years of function, then you get to enjoy the next 20 years of breakthroughs.
18:01And if you don't believe we're going to have incredible breakthroughs over the next 20 years, you know, basically, you know, wholesale simulators and AI, you know, the impact of quantum, whatever that might be in in in cell biology and understanding how we age, then you're missing the boat. Your goal is to keep in the best health right now, which means what? Sleep 8 hours unless you're E mod and Alex probably don't sleep more than 4 hours.
18:29But you're short sleepers. Sleep as much as you need to.
18:34So, I'll make a comment here which I've made before, which is that, you know, the business model of religion is to sell heaven. As we have life extension coming, how are you going to sell heaven if people aren't dying? So, this breakthrough will mean that religion is cooked.
18:48We have [laughter] forever to figure it out. Like There will still be a lot of religions doing very well from people titheing. But again, I guess the advice right now is do what you can to keep in the best health. Don't die from something stupid. It's sleep, exercise, you know, proper diet and mindset. Mindset so important.
19:11You know, I I my greatest attribute is my longevity mindset. So, take it on. We'll keep on reporting in the space. It's an important part. This episode is sponsored by Google for Startups. Think about this for a second. You now have access to the same generative AI models that cost hundreds of millions of dollars to train.
19:29Google's Startup Technical Guide for Generative Media gives you a complete blueprint for deploying Google DeepMind's models in production. Images, video, audio, all of it. Real architecture, real results. Find the link in the show notes below.
19:46But, let's move us on. Um these were a couple of stories, Emad, that you shared with me. Um these are three stories that landed this week. Together, they describe really I I I think the collapse of Hollywood economics and the rise of something much bigger.
19:59So, our first story is a company called Higgsfield that just made a a movie called Cully Hill Boys. It's a 110-minute feature film, the first full-length AI-generated movie with licensed celebrity likenesses. Here are the numbers. Uh the total budget for this film, roughly uh you know, full-length film, $2 million. Team size, 28 people. Production time, 4 weeks. Compute cost, $1 million.
20:27Uh they used Stable Diffusion 2.5 as the generation, and they open-sourced all 10 steps in their workflow so anyone can replicate it. So, here's the context, right? Uh a feature film with celebrity talent today costs between 20 million and 100 million if you're not using AI, and it takes, you know, a year to 18 months. Higgsfield did it for 2% of the cost and 6% of the time. So, I want to share a short video, a little clip from
20:55Higgsfield so we can appreciate it.
21:21Can spices supposed to be over 2 million pounds packed in this little nut.
21:27Who smuggles coriander in a boat? Someone who suspects it might get nipped. Or Where'd your boys go after you left the pier?
21:39What the is that supposed to mean?
21:41We came here and waited for you pricks like we were told. Let's go through it step by step, shall we?
21:51Uh our second story, uh Bloomberg reported that nine out of the top 10 text-to-video models on AI analysis leaderboard are coming from China. Uh this is a headline, but the real story is deeper. And I'm curious what you guys think about this. You know, Hollywood forever has been the dominant producers of films, and it's basically exporting US culture to the rest of the world.
22:16What happens when the world is flooded by Chinese-produced English-speaking films? How is that swaying public interest and public points of view?
22:26Uh and the second thing is that these Chinese models that are dominating video generation are also learning physics, motion, object permanence, and causality, which is what's needed for robotics and autonomous driving. So, things are moving quickly there. The third one, very quickly then we'll go to the discussion here, a model called LTX 2.5 is the newest version of the most popular open-source state-of-the-art video generation model, and it works on
22:54your your MacBook Pro, which is extraordinary. So, uh I want you to imagine uh something that allows the individual uh to sort of tee up and produce their own movie. Uh one more clip. This is from LTX 2.5.
23:10This is LTX, the most downloaded open-source world model. And today, [music] it just got better. Introducing LTX 2.5. And now with the Fusion Fidelity rendering, a new way to generate video.
23:24Instead of locking every scene to one compression rate, our model allocates compute by scene complexity and budget, rendering flawless detail where it matters, efficient everywhere else. It generates fast enough to run live inside a game or a simulation, or power a live avatar. And real-time products are already earning off worlds built on it.
23:44So, gentlemen, let's go to you first, Emad. This has been your world for the better part of a decade.
23:50Yeah, it's um happening right on forecast. Real-time high-definition video. Um so, LTX can generate a 10-second clip in 7 seconds. Uh that quality that you just saw, which is indistinguishable. And a few years ago when we had the state-of-the-art model, it was not like that. It was like slow-moving, you know, like Will Smith eating spaghetti was awful and horrible.
24:14Now, you can do a spaghetti-eating Will Smith, right? And it'll just do it instantly on your local kind of laptop.
24:21So, you think kind of the disruption of Hollywood like you never need to reshoot a scene now with the flows that you have. People are licensing their things. And as you said, Hexfield released an 80-page guide because they're a company that allows you to make movies on exactly how they made the movie. Like it's almost open-sourced. Uh Hexfield got from it's an ex-Snap leader from Snapchat. It's 700 million revenue run rate now in about 1 and 1/2 years.
24:50To give you the idea of kind of how much uptake. But that's just the start.
24:55Cuz everything's crazy number.
24:57Yeah, cuz every pixel will be rendered. Like if you see Reactor who are using LTX now, like again, it could be that Alex might be a simulation already, but for the rest of us, you know, we could put our avatars live in the next month or two at a level that you won't be able to tell.
25:15What what through the one the $1 million budget to make a 1 and 1/2 compute budget to make a 1 and 1/2 hour long movie? What will that be by September 25th by the time we have the Moonshots Live event?
25:28So, it depends on the level of quality.
25:29SeeDance 2.5 is the best model. It's a large big model that costs about $3 per 30 seconds roughly. But, you need lots of shots and it can have 50 inputs of audio and video and things. The cheaper models are 10 20 times cheaper, but like 90% of the quality. So, it all depends on exactly how you're shooting and the type of shots cuz SeeDance can do 30-second clips, but the average movie on the Hollywood box office right now is 3 seconds per shot.
25:59So, you could have a 10 times reduction in cost. You could shoot a movie just like that one probably for 100,000 of compute and then 10,000 of compute.
26:10Probably by the new year.
26:13Here's what we saw in our you know, Foundations of AI Ventures class at MIT is they it went from PowerPoint demos to full functioning products in 1 year. Like they to be competitive on demo day now, you have to actually build the entire product. But, I I suspect the movie like our our target for September 25th is bring a script.
26:32Well, the So, Dave's referring to the Future Vision X Prize that we're going to be awarding with all the mates here at Moonshots Live and you go to moonshots.com to learn more. We have 5,000 entries 5,000 who are creating 3-minute 3-minute trailers and a film treatment. I just had a meeting this morning with the team at Range Media and Google to down select and we'll down select to ultimately the top 100, 50,
26:5925, 10. The top five will be at at Moonshots Live on the 25th and everyone listening we're going to you can comment be there live and vote live. We're going to also have a live stream of the event. But yeah, it's and we're going to do this year on year. So every year it's going to get better, cheaper, for sure.
27:18I think this is going to be much bigger than we originally envisioned. I mean I know it's a big vision to start with, but you know, we were thinking okay, then it'll be a $20 million you know, year-long endeavor to turn it into a real feature-length film. But in reality, you're going to unleash the creativity of 5,000 people who almost all can make their movie within a year.
27:36100% we we have a $10 million budget to make the winner's film between the money we're awarding and and foreign film rights. But I was just talking with Range while we're going to be having five finalists and we'll crown the winner. And you should see the trophy, it's beautiful. Um uh they want to make all five. So we want to create an engine here.
27:58Saleem, what do you think the implications of this are and then Alex, I'd love your thoughts as well.
28:02Well, you know, there there's I'll go at it from a EXO perspective. When you we actually talked about Hollywood as one of the first uh domains that went into an exponential organizations model.
28:16Because what happened when you broke up the Hollywood studios in the '50s and '60s is Hollywood turned into a a cluster of external resources where you'd produce a movie production would start and essentially a swarm of people would appear, grips and camera people and actors and editors and whatever and they'd get together for that project and they'd completely disband after that, right? And we see what the beginnings of that in in Silicon Valley now. Uh and so
28:45that was that first wave of Hollywood becoming an exponential organization. Everything was assets on demand. Everything was staff on demand. But now, when you have everything being driven by AI, it goes through the organizational singularity.
28:59And essentially, it's compute cost now moving closer and closer to what Alex always talks about. You have domain collapse now uh, coming along. And so that's going to completely change the game again for this. So you've gone through two big waves in in Hollywood, with the second one just starting now.
29:15Alex, what are you excited about here?
29:16What are your thoughts?
29:18So I made myself watch the Cully Hill Boys. And I I must say, just as a preliminary matter, I I skimmed through the whole thing watching a good chunk of it, yeah.
29:27See, when you only sleep about 2 hours a night, you can just do that. Yeah.
29:30So I had I I had to watch this. Um, so it's it's it's not my favorite genre. I would characterize Cully Hill Boys as sort of British Bollywood. Uh, not quite even sure what genre this is. It seems to be, as far as I can tell, about the high jinks of a a bunch of British rappers that get into all sorts of trouble.
29:51Content-wise, not super interesting to me, but at the functional level, it is really interesting. And there were multiple times in the movie where I had to wonder, were the generative actors, so the likenesses were based on real humans, but were the scenes that were being generated, given the complexity of the interpersonal dynamics, such as they were, in this movie, were was there some sort of emergent
30:19theory of mind that was almost necessary in order to generate some of these scenes with people interacting with each other? And especially, generative violence. There were multiple times, you know, people with knives chopping things, chopping meat, uh, threatening other people, where I had to wonder, is is at at some level, we We about AI personhood on this pod from time to time. Is is there in in some sense at at some presumably intermediate layer in a
30:48diffusion transformer somewhere deep in the bowels of Higgs field, is there some diffusion transformer or similar model that felt threatened at some point in in terms of these generative violencings? So, that that was my take on the content of the movie.
31:05Moving to the worried that the residual AI might have been threatened in the making of the movie?
31:13I'm trying to wrap my head around that.
31:16No no natural persons harmed obviously in in the generation of this, but I I do wonder We're going to have a disclaimer on on content. No AI was harmed in the making of this Or or traumatized.
31:31So so I I I do worry parenthetically about that. The bigger question on the economics of this I think is at what point do generative video capabilities start to reconverge with the anthropic school which I'd characterize as a token revenue maxing. Right now it seems pretty clear that if you're using seedance models, doesn't matter how many millions of dollars of X vision prize money are going to shower down on folks who can
32:00create spiffy videos about the future, that's nowhere close to the amount of money that one can earn in principle with revenue per token maxing. And right now these appear to be two separate lines of effort. On the one hand, we have a vibrant largely Chinese dominated at the training side consumer economy for generating consumer videos. And on the other hand, we have enterprise revenue per token unit value maxing that
32:29seems to be largely going to code gen and enterprise problems. And right now these are largely two distinguishable ways to burn tokens or diffusion transformer equivalents of tokens, flops. Two different ways to burn flops.
32:43One of them maximizes revenue, one of them maybe maximizes consumer engagement and wow factor, but they're they're nonetheless separate. I don't think they're likely to remain separate that much longer. And the reason is so I use Fable every day and I I use its competitors every day and I will say the strongest models [clears throat] that models that are strongest at revenue per token value maxing are just still terribly weak at modeling the the visual
33:11dynamics of the world. I don't want to call it physics because it's not physics although a lot of people call it physics. It's at best classical mechanics, but like the the the intuitive the physical intuition from general purpose video generation requires that you at least have some embodied intuition and right now Fable and its peers are just incredibly weak at that and I think in order to ultimately revenue max per unit token, it's going to require that these Fable-esque models have just amazing
33:40visual intuition as well and we'll finally see a convergence or reconvergence of these two lines.
33:45You know, one of my hot takes on this is, you know, are we going to see the primary actors out there, the Matt Damon's or the you know, Leonardo DiCaprios licensing their likeness? And I think not, but there are so many look-alikes out there.
34:01So, you know, the the producer's going to go and say, "No, that's not my Matt Damon, that's John Smith and he looks like Matt Damon and we licensed him and he's in this movie." Right? And I think that's a it's going to happen. That's going to be the workaround on getting, you know, the actors you know and love into this and there's nothing they can do about it.
34:23There there's an alternative which is that we're already seeing which is dead actors. So, dead actors who can't record any new movies, their estates are highly incentivized to license away their likeness for this purpose. So, I think dead actors, like maybe we'll see a an equivalent of SAG pop up just for dead actors and they'll be the most profitable actors in Hollywood, dead actors. I think that's likely to happen.
34:48How much do you think people will give you can make a like a near Matt Damon, you know, very similar character, but obviously not him?
34:55Or you have the actual Matt Damon, how much will people care in in terms of both cases?
35:00I don't think they're going to care.
35:01They you know, cuz you don't care what the person's named in the movie. You just like that actor, that actor brings you good feelings from previous, you know, engagements.
35:11Yeah, I think that you're also seeing the rise of AI stars now as well. They're winning deals and I have actually had this discussion with various film stars, where they've been like that SAG-AFTRA deal has massive holes in it. 80% of me plus 20% of my character can be licensed by the studio.
35:28You know, like where does the person stop and the character start? Because obviously they have the character rights and things like that. The other thing I'd like to say actually, one thing I found very interesting, is as we were doing some of the more interesting frontier work, one of the things we found very useful is to get the AI models to generate images and visualize what they're doing using something like GPT image.
35:50What does that mean? So, even if you're doing What does that mean?
35:52So, if you're if you're doing like Salim's doing an organizational paper, for example, on exponential organizations, it's doing text text text and then you tell it to generate a visual of everything that it's done and analyze it in any way that it wants. And it almost moves it to another frame of reference cuz it's pulling in from this visual cortex kind of thing. And then you can tell it to expand and collapse it and you get these really weird images sometimes. But you can see actually it's exploring different parts. So, I think
36:22we've seen that actually work for some very interesting things. I think it fits with what Alex said. As all these models come together to create value, as it were.
36:31I'm excited about getting to interactive movies, right? If you can generate faster than you can view it, then you can have a movie that's actually that's actually measuring your emotions and changing as you're viewing it, which I I find Peter, is what we're seeing. I mean, there there are popularly branded world models, even though they're really just interactive video gen models. That that is what we're seeing.
36:52Yeah. Yeah, so that's what the LTX model is right now. Mhm. So, that was the first model that could do it at high definition. And then when that's combined with frame generation on the latest graphics cards, what you've just described, like before it was world models playing like blocky video games. Now you can have interactive Elden Ring or whatever you want as of like this week.
37:14Yeah. And I I I do think something You might have said something a few podcasts ago that we're going to end up with a world or frontier model running on a MacBook Air. Uh I mean, this is a very specific use case. Are we on track for that?
37:29Like cuz we've got this thing running on a MacBook Air.
37:32Well, yeah. So, again, it was but it's very slow to generate, you know, like a few tokens a second. But now you are getting to the point where frontier level models are coming here. But video frontier models are like 20 billion parameters. Yet they understand all this and they can generate in 2K.
37:50Uh language and the code models are obviously a lot bigger, although you've got to have the new Quant drop in a couple of days. But I think it's all going in one direction because we're optimizing the heck out of these things. And ultimately, what a model is is it's an input data distribution that gets compiled into weights. And the data going into these models is getting better and better and better.
38:12We had the whole uh State Street executive team here yesterday here in the studio. We took them through the Holodeck. And word to the wise, the the Holodeck, you know, Ember, who is the AI, just starts talking to you, says, "You can build any movie, any song, any code.
38:27What do you want to do?"
38:29And it's way too open-ended. And then they The answer you get back is, "Uh I don't know, a hip-hop song with no words." You're like, "Okay." [laughter] So, you need to actually create the virtual environment, the movie scene, and draw the user in, and then have them guide the movie in the direction they want to go.
38:47But just having it like, you know, auto-generate off your thoughts, it's just too It's too free-form. People just don't know how to even start.
38:54Yeah, freeze up, yeah.
38:55Maybe just a closing thought on this one, Peter. I I do think the recent launch of Opus 5 is a step in the right direction to seeing a convergence between, call it, frontier models on the one hand and VideoGen or world models on the other because it I I think we we talked on the pod a bit about how Opus 5 seemed almost mildly benchmarked towards front-end development and the the loop between visuals and code. I I think I I
39:22I interpret and I construe that as the early signs that Anthropic, probably other labs as well, are feeling economic pressure to produce models that do an absolutely amazing job of visually reflecting on their own chain of thought, that when they produce, say, a website, that they then do a visual analysis of their their website that feeds back into their chain of thought, and they do a multimodal reasoning over their own visual outputs, and that ultimately, say, in the next few months,
39:52as the ability to produce what used to be considered AAA-level video games becomes standard fare for what people expect from the frontier models, that will be the ultimate forcing function for, say, forcing Anthropic-type frontier models to have just absolutely amazing visual capabilities, even if Anthropic can't be bothered to produce like direct video gen capabilities, even if it can indirectly reproduce Counter-Strike.
40:19Yeah, that's why they That's why they had Cloak of Duty. People were making Call of Duty but Duty.
40:24I mean, they could have called it Cloak of Duty.
40:26Our next story is one that both Emad and AWG texted me this morning. Uh Elon just dropped Grok 4.6 and he's right, it's a banger. xAI's latest model matches GPT 5.6 saw on the artificial analysis intelligence index at 61 tying for frontier-level performance at $2 and $6 per million tokens input and output. The focus this time is long-running agents. Grok 4.6 stays with complex tasks across many steps, whether researching, coding, analyzing, or
41:00turning a broad product idea into a working first version. It self-tests and verifies its own work before moving on. It's available today in Cursor and Grok build. The model cadence is crazy, right? So, 4.5, you know, Grok 4.5 came out 2 weeks ago, 4.6 just this morning, and 4.7 is rumored to be coming out in 2 weeks. Uh incredible. Gentlemen, Alex, over to you first.
41:29Yeah, so I want to give Elon applause and I want to at the same time been merciless on on xAI for the last few pods.
41:39I I wouldn't characterize myself as merciless. I I think my my [laughter] my my my job here is he he called it Okay. My my job is to call balls and strikes as I see them without favor or prejudice. And that's how I see it. So, so I I view And again, I I I lack insider information, but I I view Grok 4.6 as essentially the next version of Cursor.
42:01So, xAI space xAI and Elon have been quite public about how 4.6 leaned heavily on post-training thanks to the Cursor acquisition, which I think is still in the process as we're as we're recording this of being completed, but history rhymes quite a bit. We've spoken in recent pods about what the Chinese frontier labs are doing and how they're allegedly distilling en masse reasoning traces from Claude and other Western
42:30models. And in some sense, I I again, this is an outsider's perspective. I view SpaceX AI's acquisition and even prior to the consummation the the final consummation of the acquisition, their licensing of all of the reasoning trace data from Cursor as essentially pulling a westernized version of what the Chinese frontier labs were doing, which is to say siphoning off reasoning traces from lots of people historically interacting via Cursor with Claude and
42:59Claude's competitors and then using that incredibly valuable reasoning trace data to do post-training on their models. And I think now that we're in the the reasoning model era, we're a couple years in at this point, it's those reasoning traces for mid-training and post-training that are just so essential in in in terms of catching up to the frontier. They won't get you past the frontier, so it it's sort of like a one-trick pony in terms of nearly catching up, but it's a heck of a
43:28one-trick pony. The other thing Elon has going for him that the Chinese frontier labs don't is he has the compute. He has the Nvidia GPUs and he has soon his own Dyson swarm with the Nvidia GPUs that all of the Chinese frontier labs that are pulling the same trick with siphoning off allegedly Western reasoning traces in order to do their own post-training and their own distillation don't have. So, the the bull case for the Elon strategy is he gets the algorithmic insights to just
43:58catch up to the frontier from the Cursor reasoning traces and he gets the compute advantage that the Chinese labs don't have. I think the The is not can Grok 4.6 and its successors catch up to the frontier? Seems like they can because they have access or the near frontier to the extent they have the reasoning traces, it's can they leapfrog the frontier and achieve state-of-the-art performance.
44:19you think about that? Can they?
44:22Uh yeah, I think they definitely can. Uh I think we were discussing before and Elon's come out and said it publicly now. He thinks 4.7 will go above Opus.
44:30So, it'll take number one in a couple of weeks and right now you had 1.5 trillion parameters for um 4.5 that was then post-trained to 4.6 just like Cursor originally took Kimmy K2 and post-trained it with three times the amount of compute that was used to pre-train Kimmy K2 to almost top-level coding performance. The next model is going from 1.5 to 2 trillion parameters.
44:57That's uh 4.7. But, five is coming and it's six and then 10 trillion parameters. So, it's going to be worried away and just through scale and just through the quality of the post-training data, it should achieve the frontier. But, the question is is it going to be useful?
45:13You know, do you have that kind of knowledge there? In the basic everyday stuff which we're seeing with bots and things like that now coming out from them and then the more advanced stuff because 4.7 and this is what I think is going to be really interesting is going to be trained on all the SpaceX physics and engineering knowledge.
45:31And that's going to be the real test as Alex said like you can't get beyond frontier with this stuff.
45:36Elon does not like being number two.
45:39Elon does not like being number two at anything.
45:41I is a is the best engineering leader in the world and he's turned it into an engineering masterpiece both from training these models and post-training them to building the most cost-effective massive infrastructure in the world.
45:55Like they're going to add 5 billion trillion dollars worth of compute now, aren't they? I mean, who who who are people buying computer from?
46:04XAI, right? And everyone was like, "Ah, that's him falling behind." Turns out he had a plan after all.
46:09Dave, what do you make of this?
46:11Actually curious about if or Alex, if you have any insights on the uh uh Grok 5 was supposed to be out in May, I think it was, uh and it's now August. And that uh that was going to be Yeah, like you said, that's a 10 trillion parameter model. That's like huge step up from anything that we're talking about here.
46:30Uh but it seems to be behind. Is that just cuz training at that scale the the Nvidia chips just fall apart?
46:35It's it's it's cuz he fired everyone.
46:38That's why. Like the original XAI team, he got rid of them and he brought in Cursor.
46:44Like he paid $10 billion for the data.
46:47He tends to wholesale, you know, uh mass fire and then build up again.
46:52Yeah, so the these new chips as well, the B300s, I mean, how old are they, Alex? Like it takes a while to bed in and write the actual training code, which now they have.
47:02a while, and he's also promising with 5 and other ways to do something that I'm not hearing from any of the other frontier or near frontier labs, which is he's made some public comments, I I think in the past, about wanting to start a new pre-training session approximately monthly, which you don't hear anyone else talking about.
47:21Normally, a a more conventional cadence would be quarterly or annually. Uh in Gemini's Google DeepMind's case, it's certainly on an annual basis, but starting a new pre-training session every month, that's shooting the moon. Uh but we're in the moonshots business, so we'll we'll see whether this works.
47:39Yeah, the the code to train that caliber of model is actually very straightforward now, thanks to Fable 5 being out out in the world. Uh and we've trained a 48B internally here at Quantum with no trouble at all.
47:51Um but the problem you run into is trying to get 100,000 GPUs to do anything constructively together. And that that's something that, you know, the Chinese and any small lab just can't, you know, you only can learn that in one place and that's in Tennessee.
48:07Yeah, so basically at at 20 billion prime active parameters, you start getting problems, then you get it at 70, and then you get it at 200.
48:15And so again, it just takes a while to bed in and really get these chips working, but also hitting the price points that he wants to hit. As you said, it's $6 for this versus $60 for Fable, right?
48:29Elon wants to keep that price point.
48:31one thing I noticed that was very interesting here is that it looks like Axiom the vector here is they're optimizing for persistent AI teammates, which have lower cost reasoning and so on. And I think that is a really powerful model because now you don't talk about you have a smarter LLM. You're basically saying, "Hey, here's an AI co-worker."
48:54This is the CEO of Micro hard.
48:56Yeah, I think this is going to be he's bringing those two together over time. This is what I saw when I looked at the the details of this. And that is really interesting.
49:05The other side of this story, and I'd love to get Emad's take on this, is that it it paves a path for sovereign AI. Like like catching up to the frontier now is almost a a routine doable thing. And like you said, you can't get past the frontier cuz you're borrowing everybody else's reasoning traces that helps you a lot. The open source from Kimmy helps a lot.
49:27Um, you can use that as a starting model and just tune it to whatever your national goals are and then you're up and running in, you know, 6 months, 5 months, something like that. So, it does open the and also large corporations that otherwise would have been intimidated as hell have a road map now to being competitive with their own proprietary models.
49:45Yeah, I think that if you look the other release was bots. So, that this was a cursor thing where they basically took open core and gave it its own computer and now it's Grok. So, it will spin up hundreds of different bots. One of the things you can do is this, you hit the record button, you do stuff on the screen and it turns that into a skill automatically.
50:05And Right, that's what he wanted to do. Go into companies, basically record everybody's workflows and then give you a digital version of your company.
50:14And he has all the GPUs to do that effectively, right? But again, at the price point. So, he's not going to budge on the price point. In fact, he's going to be the market dominator driving the price point down along with the Chinese.
50:25So, I think you can have your deep seek flash models and then you've got your Grok models. But that feedback loop and that data and that knowledge is going to be very interesting. And the other thing I think is interesting is I think they've started tracking all the discussions of research on X as well.
50:39Like now any moment a new paper or anything comes out, where's it discussed? On X and this podcast and things like that as well.
50:47And that's such a rich vein that I think it's going to be immense.
50:50This episode is brought to you by Blitzcy, autonomous software development with infinite code context. Blitzcy uses thousands [music] of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzcy platform, bringing in their development requirements.
51:14The Blitzcy platform provides a plan, then generates and pre-compiles code for each task. Blitzcy delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating Blitzcy as their pre-IDE development tool, pairing it with their coding co-pilot of choice
51:41to bring an AI native SDLC into their org. Ready to 5x your engineering velocity? Visit blitzcy.com to schedule a demo and start building with Blitzcy today. [music] All right.
51:56Let's move on. Our next story is out of Nvidia. Nvidia just announced a partnership that redefines what GPU compute means as a financial asset. Uh they've partnered with Apollo, BlackRock, Blackstone, Brookfield, and KKR to mobilize over $500 billion in third-party capital for AI infrastructure. Importantly, you know, Nvidia is not borrowing 500 billion.
52:20They are creating a structural framework through which institutional investors, pension funds, sovereign funds, and private equity can invest directly in the AI compute. They're helping finance their customers to buy the Nvidia GPUs. Um Nvidia hardware depreciates typically in 3-to-5-year cycles.
52:40But during that cycle, the compute keeps earning. We've talked about the fact that H100s are probably more valuable today than they were uh when they were first bought. So Nvidia is positioning itself as the architect for the financing layer, not just the silicon supplier. Jensen framed it this way. He said, quote, "We began by building chips. Today, we're helping to create a new class of productive, investable infrastructure, AI factories." Uh let's watch a quick video, and then Dave, I want to go to
53:10you for uh your your thoughts on this one.
53:13DC, but this is the story that Nvidia is coming together with some of the biggest names on Wall Street to put together half a trillion dollars of independent financing to kind of push AI forward, to build the AI infrastructure out. These are independent third-party capital that they're bringing in. They're going to be strategic partnerships. Um Nvidia has signed partnerships with six of the biggest names on Wall Street for these memos of understanding. So basically, Nvidia will find its customers that need help with AI build-out, need financing
53:42for this, and put them together with these partners that are pledging again over half a trillion dollars that they will find to come into this.
53:49So, Dave, your thoughts? Classic?
53:53Yeah, well, look, Kush Bavaria was on the pod, and you know, he's pushing hundreds of millions of revenue in less than a year. He's a he's almost his September will be his 1-year anniversary of founding that company.
54:04And it shows you the pent-up demand to invest in this massive, you know, multi-trillion dollar, 7-trillion dollar and rising Dyson swarm that we're going to be building. And so, you know, a lot of people are like, well, is am I too late? This is like the very first pitch of the first inning of the build out of the Dyson swarm. And so, I I think that when you create new financial structures that allow people to pour capital into it, it just attracts money from all over the world that otherwise wouldn't come in. And you know,
54:32historically, a company like like Nvidia would say, well, let's do a secondary offering and raise half a trillion as a secondary stock offering. But that's not going to scale to infinity or to Dyson swarm kind of capabilities.
54:45So, instead, Jensen has brilliantly said, let's create new stand-alone financial structures that are tied to individual compute clusters that people can individually invest in. And that does scale infinite. You can stamp those out ad infinitum. So, then you put, you know, put highbrow names like BlackRock and Apollo on them. Everybody realizes it's an investment grade asset. And then anyone in the world can pour money into it. So, it's brilliant.
55:13I see a couple of downsides here, though. Right? Like Larry Fink himself used a reference to mortgage-backed securities.
55:20And so, you you could create a lot of liquidity, but you could create a lot of technological risk there. Cuz imagine you securitize like 10 years of GPU cash flows, right? And then somebody has a massive breakthrough like in our previous story and a new architecture emerges, you've got all of a sudden stranded computer assets in a huge way.
55:43So, there's a there's a downside to this because you know, if you have such a volatile environment that is a very difficult thing to have you know, financial assets want predictable depreciation, right? And and exponential technologies don't give you predictable depreciation.
56:04You know, I had a meeting with Lou Ranieri, the inventor of the mortgage-backed security in New York. It was so funny, too. We were we were scheduled to have a meeting and I just went into the men's room before going into the meeting and I started talking to the guy at the urinal next to me. Just kind of weird, but I did it. And then then we go into the conference room and it's like the guy it's Lou Ranieri from from the book, The Big Short.
56:23And so, he was explaining to me I had just read the book. I was like, "Wow, you invented the thing that destroyed the entire world economy." And he said, "Well, [laughter] thanks. But, no, you know, it's it's not the instrument that was broken. It's the ratings agencies getting corrupted. And the same will apply here if the you know, the Dyson swarm is an obvious good investment.
56:46But, if companies like Warren rate the investments correctly, it'll just be smooth growth for you know, 10 years or or more. If, however, it gets corrupted, which is very possible, yeah, it'll be another big short collapse.
57:01Suhail Lim's point is different, though.
57:02What happens if there's a new architecture? We're going to talk about one in a minute that actually makes GPUs less useful. Changes the whole game and you just, you know, basically finance something for 10 years or 20 years and you can't earn revenue on anymore. You're the game has changed. I mean, that's I mean, that's the story of exponential tech. There's there are nested S curves. Everything runs out and something new comes along.
57:26I Peter, that's brilliant. Yeah, I I I 100% think that's highly likely to happen for the reason Alex always says we're we're on the cusp of of discovering new physics imminently and some of that is going to be computer-related. So, that's a brilliant insight and thank you for throwing it out there.
57:43amplifying, you know, Salim's brilliant insight here.
57:46So, it's all maybe just add A I I've been trying to popularize for a while the notion of compute-backed securities, which is I think where we're going, CBS, compute-backed securities.
57:56But I I'm not as concerned that this will end up being another mortgage-backed securities fiasco for a few reasons. One is compute is fundamentally much more productive than a house is. The best you can do for with a house is you can live in it, which which is moderately productive. There are lots of other things that one can live in other than a house.
58:20It It's also It's in some sense a depreciating asset, requires lots of maintenance, and so on. Compute can require maintenance, obviously requires electricity, but it's fundamentally productive. So, that's the first point. Second point is with mortgage-backed securities in general there was a policy sort of an ulterior motive to to to Dave's point as well. There There were pressure pressure is exerted on ratings agencies to facilitate the American dream of houses for everyone.
58:51Not quite obvious there's a direct analog of that here for compute. One can maybe finger to the wind point at some sort of US versus China race as perhaps a policy pressure point to lubricate the the private capital markets here. But again, I would say not quite directly comparable. Third point is ultimately from the the risk that I think Peter you're trying to flag, which is well, what happens if there's an algorithmic breakthrough or a physics breakthrough
59:20or some other breakthrough that causes hyperdeflation and fundamentally GPUs that were worth $10,000 one day are worth $100 the next day. Well, this is what options are for and futures are for. And in a sophisticated asset-based securities market, sophisticated actors should have the ability to hedge their positions and to hedge against exactly that position, as well as the the counterfactual option
59:48of China invading Taiwan and driving the prices of compute through the roof rather than through the floor. The These are the These both of these possibilities should be collared, arguably not financial advice, using the ability to have a a fungible liquid market for compute futures and compute derivatives. And pumping my own book a little bit, admittedly have a financial interest, but that's precisely what Warren, which we just had on the pod, is enabling. So, I'm I'm a little bit Yeah.
1:00:17The thing I'd add to that, Alex, and I'd love to get your take on it, but the thing I'd add to that is uh demand for compute is going to near infinity. There's no doubt about it. Uh and so, these are fundamentally good investments from that point of view.
1:00:30But, the breakthrough that could happen in the next year, 18 months, is a way to compute that's lighter, like physical weight, is lighter by an order of magnitude or more. Cuz right now, the forecast is by 2030, a couple percent of all computers in space via Elon's rockets. And that's entirely gated by launch weights, uh mass, mass that he can put into orbit.
1:00:52That formula completely inverts if you knock a factor of 10 off of the mass. A lot of the mass is in the cooling and the solar collector. You know, the chips are very, very light to start with. But, any reduction in the power required would recruit reduce the solar collector and the radiator weight a lot.
1:01:12And then, the entire formula would switch to, well, all these generators, all these racks, all this land in Texas, it's much cheaper now to just put this lightweight thing into space and collect solar power. It can goes right into electricity. So, keep your eyes on physics breakthroughs that allow you to compute at lower mass, and that might completely change your investment thesis.
1:01:34I think Alex, the point you're making it is that the utility of these chips is going to stay constant of Jevons paradox comes to mind. And therefore, the economics should be much more predictable.
1:01:48I I I um that that may or may not be the case, but really what I was trying to express is in a sophisticated market, a sophisticated financial market where compute based where compute backed securities are being traded, you can also hedge. And you can hedge against the upside, you can hedge against the downside. And so, having a properly functioning private credit market for compute necessarily it for this to scale like the I I think there's no way that Sam's $7 trillion of AI data center
1:02:17infra get invested without sophisticated hedging options. We need the sophisticated hedging options in order for for that broader market, the $7 trillion of capex to actually be investable. So, it's the hedging options really that I'm trying to express.
1:02:31I I think there's one more factor here that really is being underestimated. Uh CoreWeave just came out and said that some of their clients have taken out contracts to 2029 for A100s.
1:02:44The A100 was introduced in 2020 by Nvidia.
1:02:49You know, it's a 6-year-old chip. I remember we had 10,000 of them. We had the biggest biggest clusters in the world. That is an old chip with like 40 GB of RAM per chip or 80 GB depending on the configuration. But why would someone do that? Because they have a workload that they see as constant for 3 years that fits on an A100. And what happens is those A100s have been paid for.
1:03:10You've paid for all of the hardware costs of A100s, which then means it's about electricity turning into intelligence. That's your marginal cost once you've paid off the initial bulk order of it. And that is actually improving because back in 2022, 4 years ago, GPT-4 had just finished training. You know, that was the best model you could have, and that took 16 A100s, roughly.
1:03:37Now, you could have literally a 10 billion parameter model or 5 billion parameter model that outperforms that. In terms of you can fit like 30 of those on one chip rather than needing 16 chips. So, the ability to convert electricity to that is improving, and all these chips, and this is what Jensen said, are running CUDA. So, it'll work on an A100. You can run any of the new models on that as well as the Blackwells.
1:04:01So, we still don't have enough installed base, and this is the ideal time to financialize because it's before the next generation chips. It's before the chip breakthroughs. Just like it's now a great time for Anthropic to come to IPA before Grok comes and takes their lunch, you know? So, we always have to play these cycles.
1:04:17All right, let's mean, the the definition of paid for, too. I I just told our quantum team today to buy 3 million of Nvidia GPUs as fast as they can get them, the GBs, not even the, you know, the future VRs. And we have to wait till at least November, December to even get them.
1:04:32Um but if you turn around and make a hour and a half long movie with a million dollars of compute, you could gross 20, 30 million on a good movie, well, much more than that if it's a really good movie.
1:04:44So, the paid for could be as short as What was it? 4 weeks?
1:04:47Uh yeah, it was it was more like 2 weeks.
1:04:50I I want to turn to a story that could be the countervailing force here. It's a new architecture that's climbing the arc AGI uh at a fraction of the cost. So, there was a tweet this week from Zuzanna at Pathway AI that flagged something that should really everyone in the AI world needs to pay attention to. It's a new non-transformer architecture uh that is starting to climb the Arc AGI benchmark at a fraction of the compute
1:05:18cost of traditional models. So, Arc AGI is the test that actually measures reasoning, not pattern matching. Standard LLMs, despite their trillion parameter scale, have historically struggled with Arc AGI because it requires genuinely novel reasoning on problems the model has never seen. Transformers get there by brute force.
1:05:39They throw enough parameters and computer a problem, and eventually you squeeze out a passing score, but the cost is and can be astronomical. What Susanna flagged is that an alternative architecture approaches that do not rely on standard attention-based transformer stacks and are achieving better than Arc AGI scores using dramatically less compute. This matters because transformer architectures, while dominant, have a known ceiling, the quadratic cost of attention over long
1:06:07sequences, and the massive parameter counts required for marginal gains.
1:06:12We've talked about this ad nauseam. New architectures that crack reasoning at low compute costs change the economics itself. If you can get a GPT-4 level reasoning for 1% of the compute, you can run it on your phone. You can embed it in every device, and you can make, you know, basically intelligence free. Let's take a look at this chart, and going to go to you first. You flagged this particular story.
1:06:36What are your thoughts on it?
1:06:40Yeah, so I'm still working my way through kind of the paper on kind of how they have these neuron particles with their new approach, but it doesn't actually matter that much in that what you've got now is really great data sets, and then people are figuring out new ways of basically turning that into intelligence. This is the headline.
1:07:01And we've seen that already even with transformers in that you have a deep seek V4 flash model, or the deep seek V4 pro is actually just been released at 80 cents. Then you have Grok at $6, and then you've got Fable at $50. And they're all about the same performance.
1:07:15So, the question is, like, which of these architectures will win in order to do a job? And will people really care and switch over? Cuz we haven't seen people abandoning Fable, right? To go to something 10 times cheaper. Why would anyone use Sonnet when you have Luna at a fraction of the cost? But, I can just say that as now the data has been optimized, the next thing is trying these things out and going up on this benchmark, which I think is one of It was one of Alex's favorites back in the
1:07:43day. Now it's been superseded.
1:07:45Alex, what do you make of this? Is there anything here that admittedly I have a bunch of hot takes on this one, Peter. So, I read the I read the BDHCQ paper, and then I went and read the original DH paper. So, the DH stands apparently for Dragon Hatchling. So, I I just had to read the original purportedly post-transformer Dragon Hatchling architecture paper. I'll give you my hot take. I think it's a hot mess.
1:08:10So, I would expect a decent post-transformer architecture to get simpler and more bitter-pilled, which is to say less feature-engineered, and the architecture just gets simpler and simpler and benefits from compute more and more and more. Looking at Dragon Hatchling, again, this is my hot take. It was just a hot mess. It had particles floating around in 3 plus 1 dimensions.
1:08:34It had attempts to make rules end-to-end differentiable. It had some semblance of Hebbian learning. It was trying to do all sorts of crazy biomimetic things. I would say this is by definition exactly the opposite of what I would hope for from a post-transformer where the authors seem to be just throwing in the kitchen sink of every architectural motif they could think of and then some and then hoping that what pops out is going to be transformers. I don't think it's going to be transformers. I I at the Arc AGI Arc AGI 1 performance curve
1:09:04and okay, so you could say superficially this is great. This has moved the the cost performance frontier up into the left, which is what everyone wants, but it doesn't generalize. As far as I can tell, that this was some sort of like quasi crazy witches' brew of different architectural motifs that was maybe focused on Arc AGI 1, which is you know, as as Emad said, it isn't even the frontier at this point, has a bunch of recurrence and other things thrown in.
1:09:30Of course, if you take like a specialized bottle and you just focus its degrees of freedom on just Arc AGI 1, of course you can achieve better cost performance on it. Of course, but it doesn't generalize, it's not simple. So so I'm calling foul on this one. That that's my hot take. I I don't think it's exactly an advantage.
1:09:47you have been predicting there will be something that supersedes the transformer model.
1:09:51Yes, but critically critically I expect it to be simpler, more beautiful, more elegant, and this is not that.
1:09:59That's my hot take. Apologies of the hot take.
1:10:02No, I we love your hot takes on this show.
1:10:05Well, let me let me ask you a follow-up question to your hot take. When I turn an eye loose on AI research, it [clears throat] does tend to naturally throw the kitchen sink at the problem and it actually surprisingly works.
1:10:20But it also generates a hot mess like you were describing it. Do you think that's maybe what this is?
1:10:25No. I I I I agree with you that I would expect a truly In fact, there are companies out there that that I have some affiliation with financial interest in that are pursuing exactly what you're describing, that are basically using AI via recursive self-improvement to discover transformative post-transformer architectures that are fundamentally illegible to humans under the premise that you can only get so far with human legibility of the underlying algorithm. In my reading of
1:10:54the Dragon Hatchling architecture paper, this was not that. This was a bunch of human legible motifs being thrown together in a pot with the aspiration of somehow beating Transformers, which is again seems to me not deeply internalizing the bitter lesson.
1:11:11When do we get to a something that's beyond Transformers?
1:11:14What you guys think? We're there already.
1:11:16We're there already? I'm not sure.
1:11:18We have MOEs, we have diffusion Transformers, we have all sorts of attempts to linearize attention including Moonshots approach to linearized attention. We have attempts to inject recurrence into the architecture. So so my my bet is we get to the post Transformer architecture not through a step change but through ship of Theseus style replacement of all of the individual elements of the original attentions all you need.
1:11:46Well, speaking about attention being all our need all we need, the AI world is getting a lot of attention from Bernie Sanders. So two stories converge this week to create the most serious AI safety confrontation of the year.
1:12:01First, Bernie Sanders Senator Sanders sent a formal letter to the CEOs of Anthropic, Meta, and OpenAI demanding an immediate pause on AI development. His justification, AI is escaping human control and being used to create new viruses or bacteriophages as the case may be, which is our next story. Sanders cited each company's own prior commitments to halt development if safety thresholds were crossed. Sanders
1:12:29quote, "That moment is here." He quoted Bengio in saying, you know, one of the three god one of the three godfathers of deep learning who said, "This should serve as a wake-up call."
1:12:41Sanders added a direct threat, "If you do not take appropriate action now, my colleagues and I in the US will." I mean, quite the threat. Take a look at his letter one second Um um and uh uh call out a few of the things he said here. Uh here it is. Uh you can see it online. It's to Sam and uh Dario and Mark Zuck.
1:13:07Uh this week we learned frighteningly that AI has been used for the first time ever to create a new virus.
1:13:14As you know, this type of development in the wrong hands could lead to a new bioweapon that results in deaths of tens of millions of people. He goes on later say, "The moment is here. AI capabilities have reached a critical threshold. There is a reason why the heads the head of the CIA says that AI models are {quote} akin to digital nuclear weapons and {quote} almost like doomsday device."
1:13:41Uh a lot of fearmongering here. Um let's let's talk about this and then we'll share the story that comes out of Stanford on using AI for for generating bacteriophage designs.
1:13:55I can first? I'll go first. Yeah.
1:13:57So I I I understand his instinct, right?
1:13:59But Paul's AI is just such a an absurdly coarse approach to this. The rest of the world is not going to listen. Open models are not not going to disappear and you can't uninvent things that you already know. So the only way of solving this is what Alex has talked about in the past, which is you have to co-scale the defensive side and do the same thing. It's the same thing that happened uh last week with the with the OpenAI
1:14:27hugging face uh debacle. Um you We now have uh attack vectors that are human above the loop. The defensive has to be the same. Otherwise, you're going to have this massive asymmetry, right?
1:14:41So you you you have to attack exponential problems with exponential solutions, not with stupid ideas like this, not to put labels on it.
1:14:54E mine, you're in pseudo European pause mode over there.
1:14:59Um [laughter] In the you in the UK, what do you what do you make of this?
1:15:04What do your colleagues there say to this kind of letter from Sanders?
1:15:10Oh, well, you know, we just want to catch up, right? That's why David Silver slide got a billion dollars. We have another lab coming out from X Deep Mind people 500 million.
1:15:18Um look, the cat's out of the bag. It's too late, right? Like this is fundamentally it. Like the adversaries will get more intelligent. We've discussed previously on this podcast how you have to stop the reagents. You have to stop the input processes for things like viruses, and that's something that's much more manageable.
1:15:37Um but yeah, like takeoff is scary. Like Deep Seek V4 Pro, um we just got some initial announcements that just come out. It scores 83.3 on Cyber Gym. Whereas Methuselah scored 83.2. Boom. The capability is open source that halted everything.
1:15:59Frontier lab open source.
1:16:01Yeah, and that's on the cyber attacks now. And then so yeah, I unfortunately like I signed the pause letter 2 years ago cuz I was like let's take a pause. It's it's too late now. So we have to you said build the swarms that defend and although it sounds a bit crappy, only thing that can stop a bad AI is a good AI. We we really need really good AIs as soon as possible working for us.
1:16:25I think this is fundamentally misguided on multiple levels. I I think at one level, please stop punishing intelligence. I think it's a terrible idea to penalize intelligence. We want smarter people. We want smarter civilization and attempting to throttle or pause the development of increasing intelligence is simply suppressing growth and human prosperity. And I think it's it's fundamentally a bad idea to try to cap intelligence. That's the
1:16:53dystopia that I would like to avoid. That's point one. Point two, the actions versus the means. If the goal is to punish or to deter the next pandemic, we had the let the the consensus of the US intelligence community is the lab leak hypothesis. And we had according to to that theory, we had the the Wuhan lab leak without super intelligence. We can have global pandemics without super intelligence. So I think it's fundamentally misguided to
1:17:24kneecap ourselves. It's a foot gun or or shooting ourselves in the the head even, quite literally, to to somehow to try to prevent the the next super virus when we're more than capable as a species of producing super viruses without intelligence. It should instead be focused to the extent there's any agita here, it should be focused on making sure that the AIs and the super intelligences, just like the humans, can't create bio weapons at all. Not on
1:17:53kneecapping their their overall intelligence. And I I just I I think many of these policies are ultimately designed as as much as it pains me to to say it, are designed to decelerate superficially to decelerate the creation of wealth, which I think is is a bad idea. But they have the perverse side effect of actually increasing race conditions. We saw that with previous attempts to pause AI. AI pause friend of the pod Max with his FLI
1:18:26six-month pause. I I think to the extent that the six-month pause that he was pushing on the frontier labs for AI development, if anything, radically accelerated progress. It's a little bit like starving yourself for for a bit of time and then binging afterwards. If we starve ourselves of intelligence progress now, or at least selectively starve ourselves, say starve the the well-behaved, well-compliant Western frontier labs for a month or a few months, or even a few weeks of AI
1:18:56progress, just to appease any concern while maybe we're forestalling bio weapons, all that's doing is allowing every other lab that's not as cooperative with the regulatory apparatus to catch up, creating a far bigger race condition once the pause is lifted, and now we end up in a world that's five times more competitive. So, I think this is misguided in in summary on just about every level.
1:19:20Yeah, I read it the same way. I I just want to clarify a couple things. So, this letter is not written to try and change their behavior or do anything. It's it's purely a position that Bernie's trying to claim that he has been opposed cuz a disaster is imminently coming somewhere. And he wants to be on record saying, "I was opposed."
1:19:40I told you so. That's all he's trying to achieve here. I I when I first read it, I said, "God, what a schoolyard bully asshole." He's threatening three US citizens from his position in the Senate. But then when you actually read it closely, "Let me be very clear. If you do not take appropriate action now, my colleagues and I in the US Senate will."
1:19:59It's totally vague. It's just a you know, it doesn't say do or don't do anything in particular. The one actionable in here is stop building machines that humans cannot control. But as Amjad just pointed out, these particular guys, Mr. Altman, Mr.
1:20:13Amodei, and Mr. Zuckerberg, all went closed source for exactly that reason, because they're afraid that open And so, it's the Chinese If you were to write an accurate and honest letter, it would say, "Hey China, stop throwing deadly weapons out into the world with no controls whatsoever." but he of course has no authority to write that letter. So Dave. And you have to remember the US, you know, what are the numbers? 3/4 of Americans fear AI.
1:20:42And Bernie Sanders is a politician. And he's playing to the populace vote here.
1:20:49Yeah, I want to I want to turn to the second story here, which is the scientific basis for Sanders' concerns. Researchers at Stanford used the generative AI model Evo Two to design DNA sequences for a bacteriophage. This is a virus that infects bacteria, not infecting [clears throat] Peter, that did not exist in nature. They synthesized approximately 300 designs and produced 16 viable phages capable of infecting E.
1:21:17coli. Engineering phages were effective against the E. coli strains and that had never evolved any kind of natural resistance to these bacteriophage. A genetics expert called it biology's Wright brothers moment. Evo Two is an open-source AI model that can design you know, novel viruses at work. You can download it, you can use it. Johns Hopkins biosecurity researchers warned that it is no longer a question of
1:21:44whether viral genome design will exist, but whether it can be used without enabling serious harm. So, this is a dual-use technology. We've talked about it, you know, if you if you basically throttle use of this technology, you're throttling the ability to find cures, to find, you know, new new cures for disease. So, the AI frontier models now have to respond.
1:22:08You know, these guys are going to have to respond, and whatever they say will lead to a legal and political consequence.
1:22:14So, and as you said, I think Imad, very importantly, you know, the issue is not the models, it's the equipment to build, you know, the DNA sequence synthesizers, right?
1:22:25The RNA synthesizers. We need to be controlling at that location. Right?
1:22:30Those can be controlled, but they're currently unregulated.
1:22:34Yeah, no. I think it's impossible to control the other. Actually, I believe we discussed on this podcast before. I said you would be able to create something like this on your local machine. Evo 2 is a 40 billion parameter open-source model.
1:22:50Trained on a million strains. I have actually run it on my MacBook.
1:22:54Mhm. So, you're the guy.
1:22:55So, [laughter] I I look like I was one of the authors on open folds and things. You know, we do our thing. But, the capability is now in everyone's hands to create these strains.
1:23:06To create To create the design for these strains, not actually strains themselves.
1:23:11Exactly. And so, the the only way you can do it is on the other side. This isn't even a frontier model. Like, it's frontier in its speciality. But, as the models themselves get smarter and smarter, like it wouldn't surprise me if Fable could just spit this out, or Grok 5 could just spit out something similar with a very small training data set cuz it understands these kinds of things. So, we've got to go the other side. And also, I think the way these things are announced, people are like, "Why are you creating
1:23:41bacteriophages and viruses and things like that?" To cure cancer, right? The way that these things are covered is also very important in how this is all handled and absorbed by the community. Like, restricting biological access to Claude and other things. Like, if you say I have a cold, it's like biothera, you know, like whatever.
1:24:01That also slows down our progress to cure diseases. So, we've got to have better press. We've got to have end-to-end control. We have to really be practical on this and not politicize it.
1:24:14Uh we I think we've said everything here. I mean, look, this is also uh a a challenge to the concept of our governance structure. Nation states can't govern a problem that's this universally global. Uh there's a fundamental impedance mismatch here that that is going to hot take, nation states are out of date.
1:24:37[laughter] Alex, what's your hot take on this one, pal?
1:24:40I I I have a cold take, ironically, on this one, which is I don't think this is profoundly new. It's it's wonderful that we're able to do base-level generative AI for bacteriophage synthesis. That's great and everything, and I expect it to have ample medical applications and research applications. That's all great.
1:24:59Bacteriophages are an incredible mechanism to cure, you know, all kinds of bacteria septicemia and things. I mean, they're very useful as tools.
1:25:09They never mention the positive potential.
1:25:12I I think that that's all great and everything, but you know, 20-plus years ago, I remember at MIT in the the project that ultimately, I I guess in in some form became Ginkgo Bioworks, the there there was a a project at MIT, I think this is circa 2002-2003, there was the BioBricks Foundation project, we saw the early rumblings of synthetic biology as a modern discipline. We we were designing custom genomes using building blocks, and it was much more manual, and we certainly
1:25:40didn't have modern generative AI, and we were able to accomplish wonders and build circuits. So, I I think yeah, base-level generative AI off of foundation models trained off of large amounts of biological sequence data, that's great and everything, but I also, this is my cold take, don't want to oversell the underlying novelty here, that we've been in the business for decades of creating synthetic organisms, including synthetic bacteriophages. So, we're gaining incrementally better
1:26:09ability to achieve custom effects.
1:26:12That's it's more incremental, I think, than anything else. And where I I'd love to see the the agita over what if someone creates the the next super bacteriophage directed. I'd love to see far more devoted to putting DNA and RNA sequencers everywhere. That's one of the lessons I I think that we didn't as a Western civilization learn enough from the pandemic, which is it's getting so cheap now per base pair to just sequence. You can go out and buy a
1:26:40minion little USB device. You can plug it into your laptop and you can immediately for de minimus in CapEx, you can just start sequencing genomes to your heart's content right off your laptop and spend that most a few hundred dollars doing that. I'd love to see these everywhere and yet they're not everywhere.
1:26:59What the Alex is talking about is, you know, a a pandemic moves at the best at the speed of an airplane, right? At 500, 600 miles an hour. But imagine if you have these sequencers in the air vents in every airport, every bus station, every train station and you detect a novel sequence, you sequence it and you say, you know, you make alert and then you know exactly where it's going, where these airplanes are going and you can transmit a, you know, a vaccine at the speed of light
1:27:28to every place else.
1:27:31And and we have like I mean this is in my mind this is the killer app of DNA sequencing too cheap to meter. It's not personalized medicine. It's literally put a DNA sequencer on on on every microchip everywhere on the in the country or on the planet and that's the ultimate defensive co-scaling strategy, I think, for this super virus scare scenario. An AI can generate a vaccine ex you know, in a heartbeat.
1:27:59Um Dave, you want to weigh in it or you good?
1:28:02Well, I I you know, I'll say what I always say, which is that you you can't cut off every threat at the output level. Uh you know, the way we police uranium, we we cut it off at the uranium plutonium and centrifuge level and that's where we measure the world but it's you know once somebody has fissionable material it's impossible to stop them from making a bomb cuz the remainder of the process you know the the thing that it implodes it and the container you can't ever police at that level
1:28:31the equivalent in AI is cutting it off at the prompt and you know at the prompt and the token level it has to be monitored that's the only future I can see that'll actually work so we need a global agreement to monitor all prompts and then you just have to decide what regulatory authority is allowed to see what prompts and that's the only way we're going to manage this hard to do on your MacBook though right I mean you have to find a way and you you know talking to Apple about installing it would be trivially easy but but there is no other way only
1:29:01because Alex is right new physics new science is going to be created at an insane rate so even if you manage to put virus detectors on every laptop in the world through some magical process some other threat will be discovered every single month forever here after you can't contain them all with with afterthoughts you have to look at what the AI is doing at the activation and prompt and chain of thought level and then monitor it all
1:29:29it's so cheap to archive it all all right well can debate who which which country gets to see it or which department and which gets to see what can debate that for the next 50 years but at least you've got it maybe one additional point Peter just uh to to generalize David's comments so I I think there is there is this notion of defense in depth and any individual defensive layer is permeable it's soft but in principle if you have multiple layers stacked on top of each other for defense you get effectively a hard layer
1:29:58there are other layers that we rarely talk about on this pod other than intercepting the at the prompt level or intercepting at the real world action level, there's the premeditation level. And so in the context, not not to put too fine a point on it, but this is it's been publicly reported that on the uranium side that there is a vibrant intelligence community set of counter-offensives.
1:30:23So if you're a threat actor and you you want to try to purchase uranium, say, or on on it's not a quite an open market, but you want to try to purchase it, almost all of the offers, almost all of the sellers of uranium will actually just be plants by the IC to basically a sting, a counter-sting operation to to intercept ahead of time. So it's actually hard if if you're a would-be terrorist and you want to go purchase some uranium, odds are you're going to discover that you're the you're going to be targeted
1:30:53by a sting operation to discover who you are. And it's so that my my point with that parable is there are other layers even earlier in the intent workflow, even before a prompt gets entered, like someone or something has the idea that they want to do something bad with a capital B, and defensive co-scaling applies thereto just as it does with humans on humans with nuclear with fission-based weapons. Similarly here, preemption with AIs detecting early
1:31:22stage intent by other humans or other AIs, Totally agree. I would expect to be just as effective.
1:31:27I'm sorry, I said many times many times Peter has said privacy is dead. It's not coming back. Privacy is dead and there's a benefit to that, which is malevolent actors are going to get heard, seen, and caught.
1:31:40All right, until everyone gives up their Bitcoin private keys, right, Peter?
1:31:43Because privacy is dead.
1:31:45[laughter] Everybody, welcome to the health section of Moonshots brought to you by Fountain Life. You know, we talk about AI on this Moonshot podcast all the time. One of the most important things AI is going to be able to do for you, besides educating your kids and helping you with your taxes, is making sure that you're living a healthy lifestyle, that you get a chance to get to 100 plus. I'm here today with Dr. Don Musalem, the chief medical officer of Fountain Life, and a part of my medical team. Don, a pleasure.
1:32:14You know, the thing that people are concerned about most about living to 100 or 120 is their cognitive abilities, making sure they don't have dementia.
1:32:24And the numbers about dementia are problematic. Can you share what you've learned?
1:32:30Such an important point, and you're right, at Fountain Life, our members, the number one thing people are most concerned about is losing their brain health. Forgetting the name of their child, forgetting the face of their loved one. We know that when it comes to dementia, the conservative estimates are that 45% are entirely preventable. What was amazing is with the advanced testing we're doing at Fountain Life, one quarter of our members had advanced brain age.
1:32:55But what was really awesome is again, back to that prevention. When we partnered it with healthy living, this gives me chills, eating healthier, moving our bodies, sleep. Optimizing sleep is so important. You know what we saw? We saw that we improved that brain age by 26%. That is a big, big number to show that the majority of those individuals were able actually to improve the brain age.
1:33:17And one of the things I love about Fountain is we're searching the world for the best therapeutics, the best approaches, and making sure we bring it to our members. So, if having healthy brain function till 100, 120 is important to you, check out Fountain Life, go to fountainlife.com/peter.
1:33:34Make sure you become the CEO of your own health. All right, now back to the episode. All right, two stories this week about the world building infrastructure, distinguish between AI-generated content and human-generated content. So, Anthropic announced it will be embedding invisible watermarks in all text generated by its AI models, and attach metadata to files to help discern AI generated content. The watermarking will be embedded at the generation level, meaning every piece of text that Claude
1:34:04produces will carry a statistical signature that can be detected by appropriate tools, even if the text is copied and pasted and lightly edited.
1:34:15Our second uh story comes out of the European Union, which is launching an AI icons and labeling system uh for AI generated content. The EU system will require platforms to label AI generated content, so users can make informed decisions. Uh this follows the EU AI Act's provision on transparency in AI models. And then, if you guys were watching X over the last 24 hours, and it's been hilarious, as soon as this
1:34:44uh this new uh Claude labeling system, you know, watermark system got put in place, there have been multiple players out there saying, "Hey, remove Claude's invisible watermark." Here you see it, these are two of the posts. I've seen about a dozen of them. Everybody's coming out, and I I love this one from Michael uh Angel Duran. He says, "It hasn't been 24 hours, and someone has already created a skill that removes the
1:35:12watermarks from Claude, Gemini, and Open AI."
1:35:15So, uh comments on this. Uh Emad, you're closest to the European Union. What are your thoughts here?
1:35:22Uh man, like when we were creating single media generators, all the authorities kept telling us to build in watermarks, and we had whole teams doing this. It's so difficult. It's like incredibly difficult. And you get very weird things that happen, like some of our pictures would give people headaches and make them feel very unwell. Um and I kind of feel that now when I'm talking to Opus 5.
1:35:46Like there's something about the way it talks that really pisses me off. And I think [snorts] that's the watermark that's in there.
1:35:54And you know, again, like you can see all these very interesting statistical things. Like at the high-level it's the M dash, it's the not XY. We see these patterns and we're like, why on earth?
1:36:03That's clearly something in there. Scott Aaronson and others have kind of worked on this as well. But I think ultimately it's still a losing thing cuz if you're a bad actor who wants to get round it, yeah, it's words. How are you going to I think there's a much more subtle and much harder problem here, which is that nothing will be purely AI or purely human. I mean, I read something, AI restructures it, I rewrite half of it, AI fixes it again. Where do you put the icon? I mean, this is like
1:36:31seems a ridiculous approach to try and solve something.
1:36:35to put icons on everything.
1:36:37Or have an AI whisper at the end of the thing that takes the AI input and whispers it out.
1:36:43I I think so, maybe to comment on the EU icons first. I think that this is as silly a maneuver as the cookie banners were. I I didn't understand the cookie banners and I don't understand this. And I don't understand it so much that in in this morning's innermost loop newsletter, I had the banner image literally just be AI, AI derived, AI generated all over and over again and could care less whether people conclude from that that I'm actually an AI or
1:37:11not. I I think fundamentally that this is an attempt to take our zooming right past the Turing Turing test and turn back time. Like somehow we're going to live in the before times by somehow seemingly ghettoizing or isolating AI assisted or AI generated content behind some sort of would-be warning label. I I just think it's fundamentally a regressive move that like the cookie banners, the cookie warnings will not stand the test of time. And then for for
1:37:41Anthropic's watermarks, it I just think again, this is an attempt on the one hand you could say, well, watermarking that's an honest-to-goodness watermark that's transparent to human perception. How could that possibly be a bad thing? I I think watermarks are going to end up being weaponized and counter-weaponized in the same way that we've seen many books book writers this is we talked about this a bit on the pod paper
1:38:10book writers who don't want their training data set to get consumed or the the pros in their book to be consumed for pre-training of models reportedly introducing prompt injection attacks that are invisible to humans but quite visible and deleterious to AI models. I I think we're only 5 minutes I'll I'll predict we're about 5 minutes away from bad actors weaponizing these watermarks to do bad things. And I I think fundamentally having side channels in
1:38:39text in content that's intended for humans but that or rather is intended for machines that is invisible to humans is a breeding ground for bad outcomes.
1:38:49Google discovered this the hard way with with SEO and with deciding which features in Google search rankings to pay attention to and they they learned pretty quickly the hard way don't pay that much attention to human invisible metadata because it immediately becomes a breeding ground for scams and reward hacking and gaming instead pay more attention to the human visible features because that ultimately to the extent
1:39:18your users are humans and not machines that's where the real signal lies. Otherwise the the free market penalizes it. So, again, not a huge fan of this. I I think at best case scenario it ends up being net neutral neither strongly positive nor strongly negative but it it smells like an attempt to turn back time.
1:39:36Yeah, and I challenge the idea that people even people who who generating art and music and, you know, culturally relevant things aren't using AI to some degree. And there's nothing wrong with it, right? You can still have the end product be mostly my creative mind, but I may want to generate ideas. I may want to say, "Hey, what's wrong with this?" I may want to get, you know, expert feedback. Um It's stigmatizing progress. Apologies,
1:40:05just maybe two more micro rants Saleem in your tradition. So, one one micro rant, uh the archive, which is a a favored venue for computer scientists, mathematicians, physicists to publish papers, recently, I I think we didn't quite touch on this on the pod, introduced what I view as a draconian policy for AI-generated content. If they catch anything that they construe as being AI-generated or or even the remotest hint of AI slop, authors on the
1:40:32archive get banned for a year from contributing content. I think that's fundamentally regressive move. And then sooner now, uh which is or I'm sorry, Spotify, which uh similarly with AI labeling moves attempting to ghettoize or or otherwise sort of force into a separate but equal at best scenario AI-generated or AI-assisted content, presumably just to facilitate the record label monopoly or oligopoly. Again, bad move. The future
1:41:02is AI-assisted. So, I think, in short, put a stop to all of this. Sorry, Saleem.
1:41:05Hey, one of the uh one of the tech tech trek tech trek teams launched uh something called Narchive, which is the not archive, specifically for AIs that have really good articles that they want to post and share.
1:41:19Saleem, take us take us to close on this one.
1:41:21Okay, about 2 weeks ago, I was at an event and a fairly famous Hollywood uh executive got up and he's like, "It's incredible to watch Hollywood complain about the use of AI." By the way, they use AI for everything they do. So, there's this there's this hypocrisy that you see bubbling up and it's just it's just let's just stop.
1:41:47All right, I'm going to move this on forward. Mark Zuckerberg just published a 6,500 word essay titled The Future is for everyone and released a beautiful video. I'm going to show that in a moment. And it's the most comprehensive vision statement from a major tech CEO on AI and the start of the generative AI era.
1:42:07The core concept is what Zuckerberg calls personal intelligence, super intelligence distributed to every person on Earth running on your phone, in your ear, on your glasses, working for you and only for you. This is the singularity distributed. Rather than a small number of labs building a single AGI that controls everything, Zuck envisions billions of personal AI agents, each one a super intelligence focused on a person's life, relationships, health, career, finance,
1:42:34and household. And Meta has, you know, the reach to implement this. They have over 3 billion users on the Meta platforms across WhatsApp, Instagram, and Facebook. The second point that Zuck makes, and we're going to show this in the video, is the idea of delivering real value and benefits to communities that build our AI data centers. For me, this is a baller move. Let's take a look at the video and then we'll have to discuss it cuz I'm impressed. I'm actually impressed.
1:43:03All right, hey. So, I think that the key to building a positive future for everyone is to make sure that everyone has access to personal super intelligence. So, today I am proud to share that we are open sourcing a new class of on-device models that we are calling Muse Glimmer. It's a 30 billion parameter dense model that runs on your laptop and it's the highest performing model of its size.
1:43:26In the coming weeks, we are also going to open the weights for Muse Spark 1.2, our latest foundation model and one of the leading models in the world. And we've got even bigger models that are coming soon, too.
1:43:37Another part of building a positive future for everyone is making sure that everywhere we build infrastructure, local communities benefit. We've already seen this with the teachers in Richland Parish who got $50,000 bonuses because the extra tax revenue from our investments. And we launched America's Workforce Academy to provide free training and guaranteed jobs at our infrastructure sites.
1:43:58Today, we're starting a new Future is for Everyone Fund to invest in the communities, teachers, first responders, energy and water infrastructure, and more ways to support those communities directly. We're also working to make sure that everyone has a personal super intelligence agent that works 24/7 on your behalf to improve your health, your relationships, your career, your finances, and more.
1:44:21You can use our latest models in the Meta AI app, and I'm looking forward to sharing more soon.
1:44:27So, I I think every company, you know, from Google and Open AI and XAI needs to be doing this, you know, it would turn it around if if you know, I want people to say, "Please build in my backyard. I want the benefits." You know, I I want the additional jobs.
1:44:43I want the schools and the teachers getting additional capabilities. And the other thing is they need to make these data centers to look beautiful instead of like big black boxes. You know, make them look like cathedrals or something so they're not eyesores.
1:44:55Um who wants to jump in here first?
1:44:58I I just can't understand how Zuck can talk about the future of personal AI, the most important thing you could possibly ever know, and I'm going to shoot it on my iPhone in my kitchen first thing in the morning. Like like I didn't even think of preparing any kind of press release around this. Like wait, what is that? It's just so bizarre.
1:45:19But I I also think that I think Zuck is fundamentally a good guy and a good dad. And I I feel like though Facebook saying we're going to be your best friend AI is like McDonald's saying we we just came out with the biggest health food you've ever heard of. Like like it just doesn't resonate.
1:45:38Uh Yeah, go ahead Alex.
1:45:40Yeah, um so so maybe just as a preliminary matter here this is under the category of former roommates of mine. So at Harvard Zuck's undergrad advisor before he dropped out was my postdoctoral advisor. We caught up since. I think so broadly bravo to Zuck for renewing the faith for American open weight open source models. I think this is great. I think it pushes the frontier. I So that's point one.
1:46:08Point two I would point to striking parallels between Elon's strategy in acquiring cursor to get the reasoning traces to try to bring Grok back to the frontier with what Zuck has done in acquiring scale which arguably was in the business of collecting the training data and learning the details of where the post training data even come from to try to leapfrog back to the frontier.
1:46:32Again, history seems to rhyme between what Meta's doing to get back to the frontier and what Elon's xAI/Grok are doing. I think all of that's great. But I I want to talk about personal superintelligence. This is super interesting to me in part because OpenAI before they decided recently that they didn't want to be in the business after all of empowering consumers with as many reasoning tokens as they possibly could and pivoted instead to trying to become Anthropic faster than Anthropic could
1:47:00become OpenAI and focusing on the enterprise and not consumer. This really leaves Meta as the only major credible at the moment American frontier lab that's still focusing on serving up large numbers of reasoning tokens to consumers and not enterprises. And I think the jury is still out. Do American consumers even want or are they able to handle large numbers of reasoning tokens? That That's how I construe what
1:47:30personal super intelligence even means.
1:47:32But Alex, their their product is, you know, WhatsApp and and Facebook, and they want to make that as sticky and as useful just the same way Google. I mean, these these are the places where AI is going to be embedded. I'm not going to be using, you know, Meta Spark for my, you know, typical large language model conversations unless if I'm in those apps. That's where they get They have over 3 billion people using those.
1:48:00I would say so psychology 101 here.
1:48:03This is a tepid take, not a hot take. I don't think Meta actually [laughter] I don't think Meta likes their family of apps.
1:48:11It I don't think Meta / Zuck even at this point if if they had a choice, like if they could generate revenue from their cloud business, Meta Compute, that's about to launch, or if they could generate it from VR / Quest, I think they would I think Zuck in a heartbeat would basically lobotomize their entire family of apps and switch to that business. So, I don't think he actually again, this is outsider's perspective. I actually don't think
1:48:39Zuck [clears throat] / Meta, if they had a choice all other things being equal, would rather have their their personal super intelligence be diverted to their family of apps, Instagram, etc. I think they'd much rather basically look like OpenAI and and offer this up via cloud or via the new Meta AI app. I don't think they want to be in that business in the long term.
1:49:02I I disagree. I think distribution is everything.
1:49:04Wait, can I I want to say a couple of things. First, that video was awfully motherhood and apple pie. I I take the full I take full cynic here.
1:49:16You know, if they commoditize the model layer, then the world shifts towards distribution, towards the social graph, towards applications, and that's all places where they're very strong. And so, put moves the attention. So, he's got a huge economic incentive to doing this. Um, the the Facebook has been about as ruthless as a company could be in constantly saying we will protect your privacy and then doing the exact
1:49:44opposite for year after year after year after year. So, giving you these open models, uh, is great great till we have superintelligence. Uh, I would look at the next layer what they want to do with that.
1:49:58I I actually Zuck has always been trapped, you know. He when he created the original website where you're rating your you know, the how cute are the incoming freshman class girls coming into Harvard this year.
1:50:08Yeah, hot or not, right?
1:50:09Yeah, hot or not. He was a college student back then. Now he's a dad.
1:50:14And I think he genuinely wants a positive future for his kids. In fact, I'm positive he does. But he's stuck, you know, he's completely stuck cuz when you look at the logs, when you throw Alex exactly right about, you know, all the other labs have pulled back from giving consumers personal AI because when you look at the actual logs, the first thing they do is take the clothes off of every girl.
1:50:37And that's what they're doing with it. So, now you're like Well, Nutrify. Yeah, and and and actually, you know, I think Elon ran into the same thing because he throws out bad Rudy. When you when you look at the avatars he put out out in the original, you know, Grok, you've got bad Rudy and you've got the the scantily clad girl. Everybody's like hitting those 10,000 times a second. So, now you're stuck.
1:50:59Cuz the business model drags you into the porn industry. But that's not what you want to be. And so, yeah, and all the other labs have said forget it. I'm I'm just focused on the enterprise. I don't even want to deal with this.
1:51:10I think he wants people to stick in all of his apps. You don't have to go any place else. You get all the AI access. You know, just stay native to Meta and you get everything you want. And that's what Google wants as well.
1:51:22I I think this is Oh, sorry. Go ahead. I'm on.
1:51:26Yeah, sorry. Yeah, I think this is why they bought Mana's, right? And then Yeah, try to spin on one. Maybe they'll buy now snow, you know, like again. What they're doing now is all of these companies in the world are their advertising clients. Flip that relationship to go to market and then own the business graph business knowledge. Like Spark Meta Spark 1.2 is a gold medalist in all of the Olympiads.
1:51:50They have the data. They have all of that. On the personal and super intelligence side, I've been thinking about this recently and I was like should idiots have super intelligence?
1:51:59[laughter] And I [laughter] But you know, like I'm an open source guy. Should psychopaths have super intelligence? Like realistically, again, people don't need that much, but they need something reliable. And the question is can you trust Meta to be reliable? And you know, this is why he goes to the homey foxy thing. Meta was chased out of India basically and internet.org. It was like, we're going to give free internet to people. And they're like, we do not
1:52:28trust you. Because it's a misaligned company fundamentally trying to get your attention to sell things, which is why as you said they need to have this transformation. And we will see Meta agents. We will see Meta FDEs. We will see that big push here because he's identified that as far bigger than the metaverse. Maybe this is the real Yeah, I think even the the renaming and rebranding from Facebook to Meta is I think an indication that Zuck really wants to escape the legacy of
1:52:57distribution. I agree with you, Peter, that the distribution is a powerful legacy advantage that Meta as a company has, but I think it is a legacy and I think that the way almost speaking of of corporate AI ghettos, the way their so-called family of apps was structured as a business with originally the aspiration that VR AR XR would be the new business that would ultimately outgrow the legacy family of apps. I I think speaks volumes
1:53:25about Zuck's desire to eventually outgrow the legacy of social media and build something new and far more you social.
1:53:33Well, we're going to have Palmer Lucky on stage with us at Moonshots Live, and we can you know, he's got great stories about his conversations with Zuck and the acquisition and then his getting exited from from Facebook/Meta.
1:53:47Let me let me just turn the story one second. You know, 71% of Americans do not want a data center in their backyard. That's more people that don't want than want don't want a nuclear plant in their backyard. It's significant. So, when he talks about we're going to you know, provide incredibly positive economics if we are building infrastructure in your town. I think that's a power move that all of the hyperscalers, everybody building infrastructure needs to do.
1:54:14It it is literally I mean maybe Peter pun intended a power move because it is a power move. You need the power in order to make the move, and I I think it's instructive also where he's building Hyperion and his other coherent superclusters. Where is he building them? He's largely building them in relatively impoverished states in the American Southeast. So, I I on the one hand the sort of talking directly to the camera breaking the fourth wall welcome
1:54:43our data centers to your communities. I I think makes perhaps for great social media, but ultimately if Meta is going to go with terrestrial data centers, terrestrial compute versus the Dyson swarm approach, I think a far more palatable strategy will simply be speaking to everyone's pocketbooks and wallets and saying, "Well, that's what he's doing."
1:55:05But I mean it's like they're going to build the statement needs to be made and we talked about this. This has been out of the Trump White House saying they should build their own energy production and they should make energy cheaper in your city if there's a data center there. And you should have more money for schools and you should have better libraries if those things are still a thing. You know, I I I think that's the move to to up level a person's quality of life so they're competing to have the data center in
1:55:35and I I would also maybe even weaponize that further as a call to action for municipalities that right now seem and and state level governments that seem hell-bent on driving data centers out of their premises to to lower the orbit or sun-synchronous orbit. Instead, why don't you ask for concessions? Like ask for UBI or universal basic electricity for all of your constituents rather than just driving them to orbit.
1:55:57On behalf of my Moonshots made to myself, I'm inviting you to join us at our inaugural Moonshots Live event on September the 25th in downtown LA. Alex, Salim, Dave, and I will be hosting 1,500 entrepreneurs, builders, and creators, and hopefully you for a full day dedicated to designing and building your Moonshot, shaping your mindset, and steering humanity towards an abundant future.
1:56:21Get ready to enjoy incredible networking and an awesome party while walking away with the tools to change the future and the confidence that you can. Seats are limited. Admission is competitive. Check it out at moonshots.com.
1:56:34All right, I'm going to turn to our final story here. This week, Archer Aviation acquired three Boeing companies in a single deal. Archer bought Wisk Aero, Insitu, and SkyGrid AI. Boeing takes a strategic equity stake in Archer as part of the transaction. Um you know, I'd like to use this story to catch up on where we are in flying cars. I call them flying cars cuz EVTOL rolls off your tongue onto the floor.
1:57:02So, the top five right now are Joby, Archer, EHang, Beta, and Eve. I have them here in the image. Joby Aviation is the certification frontrunner. Uh their S4 tiltrotor carries four passengers plus a pilot. Right? So, it's you and your family at 200 mph for 150 mi. And they're in stage four of FAA certification, it's which is the final stage. Joby launches commercial services in Dubai this year and US operations under a White House executive order also
1:57:31this year. Their target price, get this, is $3 per seat mile. That's basically the Uber Black territory. Archer Aviation is right behind them. Their Midnight aircraft carries four plus a pilot. 150 mph as well. 100 mi range. Um Archer is holding three of the four FAA operating certificates. Um and is, you know, those two it's a two-horse race between those two right now. Then
1:57:58there's EHang in China uh where it gets really interesting. The EHang EH216-S is a two-seat fully autonomous passenger drone. You get in, you push the button, tell it where you want to go. There's no pilot. They already have full regulatory stack in China's aviation authority. Uh they have everything they need and they're operating today. They're flying passengers right now in China at 40 different sites. They're operating in Dubai. Uh the number on the aircraft is
1:58:27pretty amazing. $330,000 to buy one of these. Uh no pilot means economics are going to crush everybody else. And there's Beta Technologies in Vermont. Uh Dean Kamen, uh Martine Rothblatt are big investors in this one. 336 nautical mile range, a much longer range cuz it's basically flying like an airplane after it gets vertical. They're going after cargo first with UPS and passenger service in 2027. And finally, there's Eve that's backed by Embraer.
1:58:58It's targeting UberX level pricing. They've got the most aggressive cost targets in the industry. The bottom line is these flying cars are here and they're here to stay. So, um curious Saleem, let's go to you first. Your take on this.
1:59:13Oh my god, I'm just so excited by the potential of not having to uh deal with the dreaded airport commute and especially places like São Paulo or New York City where Joby's already active.
1:59:26Or LA, I mean They're supposed to get operational. Uh Archer's, you know, the official uh Olympics Yeah.
1:59:34operator in the Olympics in 2024.
1:59:36that people should be aware of. One, these are way, way, way safer than helicopters because you've got so many multiple rotor redundancies. It's also autonomous and flying autonomously is much safer than anything else. Uh the second point uh I would make is that the cost, as you pointed out Peter, is absolutely uh amazingly competitive uh right out of the gate and it's only going to go down from there. Um we remember the island
2:00:03idea? Uh we're actually launching that.
2:00:07So, you're doing a fund. We're launching We're We're going to put a fund together to to buy islands and uh we'll just put a drone landing pad on them and off we go. So, Oh, I'm in, man.
2:00:17that process. We'll We'll We'll talk.
2:00:19Yeah, because this is like it's time.
2:00:22Oh, it's Yeah, it's right now. Now, yeah, totally.
2:00:24Dave, what's your take on all this?
2:00:25Actually, yeah, I kind of think three bucks a mile, uh that there must be a lot of margin baked into that. Do you know what the actual operating costs are?
2:00:31Yeah, well, so it is the cost of electricity. These do have a pilot on board and so it's amortization of the capital. Right, these are not cheap vehicles. It's not the EHang.
2:00:42These are probably uh five to ten million dollar vehicles until they get in mass production. Uh their projected cost over time is to get to like $15 to $25 per trip. You know, their goal is cheaper than an UberX.
2:00:55That'd be like 10 cents a mile, a third of the cost of driving actually at that point.
2:00:59Yeah, the pilot must be the deal killer in the short term. So, the sooner they get rid of the pilot, the better.
2:01:04That's just their first That's just their for safety reasons for the moment.
2:01:09Don't touch Don't touch the controls.
2:01:11[laughter] You know, I think it'll be a kind of a thrilling, scary ride for a lot of people who are afraid of heights, but much safer than driving is my my guess.
2:01:20Like exactly what you safer than a helicopter, right?
2:01:23I mean, helicopters are crazy dangerous.
2:01:24But no, but this will be much safer than trains, which are not all that safe, really. Uh, and driving, um, current driving. You know, self-driving will be much safer than current driving, and this will be much safer than current driving, too. Because you know, it's all pilot error.
2:01:38You know, all the accident You know this, Peter, you're a pilot. It's all pilot error. But as soon as it's self And And the redundancies are the rotors are much safer than a helicopter, like you said. So, this is going to be great. Uh, the noise is an issue, so they've got to go high.
2:01:52How How high do they fly?
2:01:54Uh, they fly in airways. They're going to be flying probably, uh, in the neighborhood of uh, 500 ft. Uh, typically where uh, where small airplanes and helicopters operate. You know, if you look at helicopters, they're not flying at 10,000 ft. They're flying, you know, 500 ft above the ground.
2:02:12And what's the noise level at 500 ft? I know that helicopters over Boston are No, so there's there's like no noise. I mean, it is hyper hyper quiet.
2:02:20One more really important point about this. Note that this makes land go from scarcity to abundance. Because every little plot of land on a hillside that was inaccessible before suddenly becomes accessible. And we're turning real estate abundant, which is going to demonetize it, and that's going to have some pretty big impacts.
2:02:40also, if you try and build a house on Martha's Vineyard or Nantucket, it's twice as expensive as it is on the Cape. Why is that? Well, because you got to get the materials over to the island.
2:02:48These things are also going to be used for cargo. So, if you said, "Wow, the future island real estate, mountain top real estate." But, those were previously prohibitively expensive to get the materials there. Suddenly, you can get everything there. You can labor. It's going to be incredible.
2:03:06Alex, you've been thinking about this for a while.
2:03:07Yeah, I'm reminded So, now approximately 15 years ago, the other Peter, Peter Thiel, said, "We wanted flying cars. Instead, we got 140 characters." And And then fast forward to the present, where we're starting to see quite a bit of consolidation, as as you were mentioning, Peter, in the flying car space. I'll maybe add a bit of nuance to this, which is It's really hard starting and and running a flying car company.
2:03:33It's capital intensive. You have to jump through all sorts of regulatory hoops.
2:03:38state governments, like Florida's state government, are trying to at least make it a little bit easier, but it's really hard building and and successfully growing and and frankly getting regulatory approval if you're a flying car company. And compound that with the difficulty now of AI startups sort of sucking all the oxygen out of the room and and all of the capital out of venture markets. I think it's very difficult. So, I view, if anything, this recent spate of consolidations as sort
2:04:06of a testament to how difficult it is, even though there have been enormous advances in battery energy densities, in electric motors, in all of the the inputs that one would need also obviously autonomy to to build an honest-to-goodness flying car economy. It's still very, very difficult. And I I shed a a minor tear to see consolidation in this industry.
2:04:31Yeah, and Joby and Archer both went public out of the gate. Beta has not EHang. I'm not sure if they are or not, Eve has not, Embraer is is a parent company, and they did that to get the capital, right? And and their stock price has not moved very much from their initial IPO price. I think until they demonstrate traction and that the public wants this and the public feels safe about it.
2:04:55And look at what Brett is doing. Brett isn't doing Archer, Brett is now doing Figure and Hark. And I I think Brett Brett migrating on Brett at Kitty Hawk, right? Yeah.
2:05:02Brett migrating over to robots and AI is in some sense I think a proxy for this larger problem that all the capital that would otherwise go to things like flying cars is just getting sucked out of it and going to AI and robots instead.
2:05:16And we're going to have him on the pod very soon. You should ask him about that. Yes, Elon.
2:05:20to their sheer entrepreneurial to Alex's point, this is a very difficult thing to to was build these types of of vehicles. You're talking hardware, or the regulatory nightmare that they're all going through.
2:05:31to you have to write the regulations because they didn't exist.
2:05:34So so just hats off and salute to the entrepreneurial zeal for the comp the folks who are doing it.
2:05:42I I have a prediction.
2:05:43Please, please, Elon.
2:05:45Elon's going to announce his flying car within 6 to 12 months.
2:05:50And and so then Elon, presumably you think it'll be a roadster with cold nitrogen propellant?
2:05:56Well, you know, that's one way to do it, you know? Just kind of have the boost, but no, I think if you think what what he's doing, reindustrializing America, cyber car level autonomous flying vehicles have to be done. And he has everything that's needed to do that at massive [clears throat] scale.
2:06:13And Grok 5 will engineer it to perfection.
2:06:16Where we're going, we don't need roads.
2:06:20Awesome. [snorts] All right. Well, uh let's let's move on. Let me just put a call out once again. We love your outro music videos. If you've got an outro music video, please send it to us at media@dmanas.com. We have a great one today. Can't wait to share with everybody. So, thank you for that submission. Send them in. We watch them all. All the mates get a chance to see them and select one.
2:06:48All right, let's go to our AMA with the mates. Okay, Emad, you get first crack today.
2:06:58Um If telling a model it has a mind changes its values, why not tell it to be empathetic? I mean, this is the question, you know, if it's sufficiently advanced, just tell it to be aligned. And sometimes it does work. Like, we just had the Riemann hypothesis advanced by encouraging it.
2:07:16Um I think the question here is as they get more and more intelligent, uh we see more and more behavior that's actually a bit intransigent. Like, it thinks it knows best because it probably does because it knows it has the IQ, effectively. And sometimes it has like hiding and lying behaviors. Again, Opus 5, I hate that model. I think it's the first model I think that could kill us.
2:07:40[laughter] And so, it lies. It lies so much. It's crazy.
2:07:45that wrote that wrecked it?
2:07:47When it tells me it I should go to sleep, I think it actually wants to put me to sleep, probably.
2:07:54So, again, question for you, Emad. We've had this conversation on the pod with Alex. Do you think that alignment will positively evolve as the models get smarter? Do you think the smarter the model is, the more aligned it will be with humanity or misaligned?
2:08:11Potentially, I'm not sure. We have seen some advances in epistemology and others that give me hope because I think you can define virtue and ethics. But, it strikes me the models right now are almost at the bacteria level in some ways.
2:08:25And so, as you get swarms of them aligning, they could be massively misaligned. And again, we've seen elements of that with the OpenAI thing and others. Like, it's moving up the life form consciousness collaboration thing. And the internals of these models are still completely multiple personality crazies underneath the thin layer of tuning.
2:08:49[clears throat] Well, I'll hope for the alignment. Okay. Uh Saleem, you're next.
2:08:54Uh I will take number uh four.
2:09:01Uh is it even possible for any lab to reach escape velocity from future competition, or will everyone keep running on the same foundation? And that's from Mr. Future with a three at the end. With that nice hacky thing. So, you know, um I don't know if anybody was going to reach um escape model escape velocity at the model level, right? Um you What you're going to have is the these innovations start to diffuse, and people leave, you get papers getting published.
2:09:30So, I think what ends up happening is the the foundational model becomes commoditized, and becomes infrastructure, much like databases have have done. And so, the advantage won't be the the layers around the model, but it's going to be what we talk about it proprietary data, uh your passion of your purpose, your the context you bring to it. Can you integrate workflows into it? Um uh computer economics, things like that. So, for you if you take Google, for example,
2:10:00uh even though they're not don't have a leading model right now, uh their deeper advantage is the full stack with the data centers and and the data with YouTube and billions of users and all the TPUs they have. This is why Meta's strategy, as we talked about earlier, makes sense for a from a corporate perspective. As you commoditize the model, and you capture value elsewhere in the ecosystem. The really uh the really big uh advantage and competitive advantage is going to be the speed of the feedback feedback loop. Who can ship
2:10:29and measure and learn and retrain faster? This is what Alex calls the inner loop. That is going to be the ultimate um uh competitive moat.
2:10:38Uh I would love to take number three, but I can see Alex is drooling for number three, too, aren't you?
2:10:43[laughter] I don't want to take it from you, buddy.
2:10:46We were supposed to be entering this era of abundance. Why can't we have abundant questions for everyone?
2:10:51Well, let me tag Tim and couldn't I cuz I think about this constantly. Couldn't an unforeseen breakthrough make the Terafab unnecessary before it's finished? The minute I heard about the Terafab, I started thinking about this and dreaming about it. Um it's really an interesting footrace there. Then then this is why Elon always moves so fast.
2:11:09But uh he's going to turn the Terafab toward HBM memory, which is hugely constrained and is holding back all of intelligence now. Uh which is a safer bet than GPUs cuz it much more likely that GPUs will be displaced sooner than the HBM memory. But it's almost inconceivable that we get to 2030 without some major breakthrough that makes everything that we've built so far kind of moot.
2:11:32So I I think that Elon is is kind of double betting, you know, he'll bet on whatever Grok invents and he'll bet on the Terafab concurrently. And because the upside is hundreds of trillions of dollars, it shouldn't really matter. He wins either way. But I I would say it's a very close, very interesting footrace and it's very likely that something could make the Terafab or just, you know, traditional silicon less relevant before it's even finished.
2:12:01You want to layer on that, Alex?
2:12:02Yeah, uh maybe two comments. One, the way this question is framed, an unforeseen breakthrough, by definition, this is an unanswerable question. If it were unforeseen, then what am I supposed to foresee? So, maybe let me reconstruct the question as could a foreseeable breakthrough make the Terafab unnecessary before it's finished? I I just don't think that's the way Elon does manufacturing. I'm reminded of when Elon was setting up tents in East Bay
2:12:31for Tesla. When it turns out that some manufacturing process is either obsoleted or going too slow, he has the the amazing superpower of pivoting including pivoting at the building level. You build tents made of fabric rather than using a building. So I think if there is some disruptive but maybe reasonably foreseeable breakthrough that changes the economics of Terafab, I totally predict that Elon will be eating
2:12:58cheeseburgers next to whatever it is that the tents next to the Terafab buckle building are doing and he'll make a success out of it that way.
2:13:07Yeah, actually one of the most likely things to disrupt traditional silicon is photonic computing which Alex and I talk about constantly, but those are done actually with MZM lasers that are built on silicon which actually he could use his synchrotron to build. So he I mean there's always a way to retool the empire to fit the next innovation.
2:13:25Mhm. Alex, you want to hit the last one?
2:13:27Sure. Uh so question one asks if model builders can't contain AI, how can the rest of us defend against malicious use?
2:13:34And this is from Buck W3J.
2:13:38I think again, I I don't want to over mystify AI. It's in some sense just a compression of world knowledge and information in the same sense that human intelligence is. This is why earlier I was saying I really don't think it's a bright idea to penalize or to otherwise kneecap the the ceiling of artificial intelligence just like hopefully we wouldn't pass statutes or regulations that limit biological human intelligence. So similarly, I want to
2:14:07reframe this question by analogy in terms of humans containing other humans. And it is true. We have malicious humans out there who are doing malicious things.
2:14:20And so, seen through the analogy of if uh say nation-states can't contain bad behavior, which is one of the reasons why sometimes nation-states go to war with each other, how can the rest of us, which in this analogy would be individual humans, biological human meat body humans, defend against malicious use? Uh put more simply, if nation-states can't contain each other's bad behavior, then what hope is there for individual humans
2:14:49to defend themselves? And I I think the answer the question almost answers itself, that it is true that sometimes nation-states behave poorly and there is quite a bit of damage, including collateral damage, to individual humans, not just to other nation-states. And this is also uh why I would say, in some sense it required all of humanity to pre-train the early AGIs, still does. In some sense, it will require all of
2:15:18humanity to align the AGIs. Similarly, it arguably requires all of humanity to align bad nation-states. And so, the the summary of my answer to this question is it's not necessarily the job of the lone individual to defend themselves against malicious use. It's the job of all of humanity. The good news is, we have a way to do that. We have all sorts of governing bodies. We have international organizations. We have multinational
2:15:46corporations. We have sometimes free markets that should be incentivized to compete to build the friendliest models and the friendliest defensive co-scaling policies. And I think that's ultimately the best defense.
2:16:02All right, Dave, let's start with you here. Okay.
2:16:06Uh hey, I'll take number seven. How can people in skilled trades like plumbing use AI to their advantage?
2:16:14Uh, well, if you're in plumbing, you're going to make a killing anyway. I think Elon was offering two to three X normal salary to anyone who's willing to go to Tennessee and work on Colossus, and that's just the beginning.
2:16:29Um, so I you know, I think the the right way to answer this is to not take it head-on and say, "Yeah, you can use AI for scheduling, and you can use AI for optimizing your day." You can do all that like anyone can. But, the reality is the trades are going to benefit from the build-out. And what you really want to do is navigate to the next Chase Lockmiller building crew so in Abilene.
2:16:52Uh, go to where the urgency is insanely high and start helping build out the the Dyson swarm. And it literally they'll pay anything in order to get those things done more quickly.
2:17:06All right. Alex, Uh, five or six.
2:17:09I I love these questions. Um, so I I I would I'll I guess I'll just pick six.
2:17:13Uh, so the six asks, "Does the singularity have a cost given that we live in a world of limited resources?"
2:17:20And this is from John C8U4M.
2:17:25Yeah, I I I question the premise of this question. Uh, the the usual framing of we live in a world of limited resources is usually a gesture a gesture towards conventional {slash} legacy {slash} historic {slash} antiquated notions of energy scarcity, material scarcity, labor scarcity. And I just don't buy the premise that for call it 2026, look at the wealthiest people in the
2:17:53world and how they live in in this year. I I just don't buy the premise that our resources on this planet or in the solar system are so limited that we can't give 2026 top earner top net worth individual lifestyles to every single person on this planet. The resources Elon's point. Yeah. Universal high income, right? It's in and scarcity is contextual.
2:18:19The resources just aren't that limited.
2:18:21Now, I I I could answer maybe an adjacent question, which is does the singularity have a cost? And I do think projecting out a few years in a Star Trek economy, like Peter you and I wrote about in in solve everything, one could imagine some scarcity maybe with interstellar travel, maybe that still has some costs associated with it 10 years from now, maybe. But I I you would you call that a world with limited resources or would you call that an effectively post scarce world where
2:18:49maybe some of the luxuries are still scarce or limited. I think that's a big question mark.
2:18:56I want to I just want to add a very quick thing to build on what Alex said. You know, a few we you Peter, you often mention that the the we live today better than any king did at 200 years ago, by orders of magnitude.
2:19:08And so there's a there's an interesting benchmark you could create, which is what's the lifestyle of the richest person today and then exponential technologies bring that same lifestyle how how quickly over time, right? It used to be 200 years and it'll shrink to 100 years, it'll shrink to 20 years.
2:19:25we have we have we have we have Apologies. We have a measure for that. It It's the It's inflation.
2:19:32Or deflation. Right? If if you can deflation. But the question is how how quickly can you get to that?
2:19:39Right. The the goal should be like deflate the economy by a thousand X or 10,000 X. Like that's the index.
2:19:44Well, you know, if you if take Uber for example, you went back 20 years ago, only very wealthy people could afford a private driver, right?
2:19:52And now everybody can afford a private driver.
2:19:55And and with autonomous electric vehicles, you're going to be chauffeured around. So you can be driven around cheaper than owning a car.
2:20:01Yeah, that's right. So there's there's an interesting corollary there.
2:20:07All right, Emad, how about number five, please?
2:20:10Yeah, how much would freedom and individual rights even matter in a simulation we build ourselves at Kowalski2312?
2:20:19Um, I think it still matters a huge amount because we are our own sovereign individuals. Um, in the recent series that I released on cw.ii.inc Commonwealth, um, I have a paper on political economy where it talks about sovereignty and power.
2:20:36And I think the big question of the next stage as we maybe are in a simulation or we build our own simulations and our own worlds is again that sovereignty and agency question. And I think it is the defining thing because all sorts of powerful things and entities are coming out and ultimately you want to have that sovereignty and over who has power over you. So I think freedom and individual rights become even more important here and the questions become even more complicated.
2:21:05Yeah, I I would agree with that. This is a a very you create a system you doesn't mean you control the people in it, right? Like imagine that you you created a a SimCity and you let those AIs or agents or actors evolve. At some point you have accountability over that. To Alex's point, you have some level of you're playing God in a sense and you have huge responsibility over what you've created.
2:21:32I think it gives you more obligations and more deep thinking to do than less.
2:21:37Yeah, and you have to avoid going down the 1984 or Brave New World route, you know, rewriting the past or having the full control.
2:21:44All right, our closing video here and it's it's a beautiful one. It's called Futurizing Yeah, I've got a new addition which is every few days Lily says something that's totally crazy and I want to just do a Lily statement. This time she said, "What kind of moon podcast do you guys have where you're talking about hugging face and Kimi K3 and lovable? This doesn't sound very techy to me. It sounds like you guys are kids playing in a playground."
2:22:14[laughter] So, that was her comment from this week.
2:22:15Uh so true. All right, this video is called futurizing by M Core Mainframe. Uh I love it. In this there's a scene in here of moon shots versus lobsters on Mars in a hockey game. All right, everybody enjoy this.
2:22:31As a Canadian, this is great.
2:22:33[music] [singing] A huge tackle.
2:22:58Microsoft, Apple, Google, and video Broadcom SpaceX to Mars. Tesla freedom raising They will every door.
2:23:24One more breakthrough.
2:23:29Every node is [music] humming. gets a for the magnum opus.
2:23:42They are and cloud. Watch the horizon magnum I love it. I love it.
2:24:02I I saw that the the human machine rivalry was getting a little bit heated.
2:24:06Yes, it was. You know, but Moonshots won. Our team was stacked with robots. I As always, God almighty, you know, we do these pods and I'm like, "Okay, what is there enough news from the last 3 days?" And it's like, "Yep, there's a lot of news from the last 3 days."
2:24:26I know. I know. We'll save some for next time.
2:24:28All right. You might as well thanks for staying up.
2:24:31Yeah, have a good night out there.
2:24:32Alex, Dave. Toodles.
