0:00We're going off script. Just having a beer.
0:03It's not the first non-Guinness, but it is the first IPA.
0:07There's a first time for everything. Cheers.
0:12I want to ask about PHP week.
0:15Yeah. First off, how did it...
0:19Why did it take so long to find these bugs?
0:22Because it sounds like there were— You mean why did it take the 20 years, or why did it take the last few months?
0:25No, why did it take... Exactly, why did it take a year?
0:28Maybe start by explaining the bug. Or the problem.
0:31So the problem is that although we have known for a while that these models are getting good at finding vulnerabilities in the wild.
0:40A) They continue to get better fairly rapidly, and B) People just hadn't really used them to go look at everything out there.
0:48There's a lot of surface area and a lot of code on the internet.
0:52And, as people have started to do that, I think we're going to find that we are in a world of many more serious vulnerabilities than we expect.
1:02Even though many people who have been following this for a while expect a lot.
1:05Yeah, I just feel like a lot of the discussion ends up being around we need to fix the software.
1:10And sure, yeah, we should fix the software.
1:12But also, we maybe need to change the architectures, which is harder, but a much more durable fix.
1:18That seems to me what needs to happen. Yeah.
1:20It would seem really hard, except we have this amazing new tool to do it with.
1:24So, I kind of think that's what's going to happen.
1:26And you're already seeing, like, a new generation of startups talking about building this new kind of internet or new kind of application architecture for this world where you assume that the models get smarter and smarter.
1:38What do you think the global software upgrade looks like?
1:41For example, is a PHP codebase just a bad idea and you should rewrite a nice, clean version in Rust?
1:47Or, do we upgrade PHP?
1:50Everyone thinks that we need to kind of upgrade all hand-spun software.
1:56Just what form will that actually take?
1:58Is this huge for Infosys and all those guys?
2:01Because, it's going to be a lot more software consulting projects.
2:04I mean, people are talking more and more about just, "Let's re-architect what it means to be a router, or an operating system, or an application." And, you know, I've seen some of these new startups saying that they're just going to make a whole new stack.
2:23And that would have seemed insane two years ago.
2:26You would have been like, "There's no way. There's too much surface.
2:27You can't write all that." And now maybe soon you can.
2:30I would bet that we have a completely new sort of model of how we make systems and networks.
2:37What else reminds you of the early internet days?
2:39The thing that most reminded me of the early internet was...
2:43There is this spirit of just unbelievable, creative unleash in people using these tools.
2:52You see it in kids particularly. Mm-hmm.
2:54And again, maybe this is just because I remember what it was like when I first got a computer with internet and I could do all these things that adults just couldn't even think to ask for.
3:03And watching an 8- or 10-year-old make a complete video game, with some crazy ideas they have that no adult would really have.
3:12And expect that to just be the way that technology is going to work for the rest of their lives. In fact, getting better and better.
3:18And, just this constant "I'm going to make useful stuff for me, and show it to my friends."Yes.
3:23We had to suffer when I was a kid to make a piece of software.
3:26Yes. That barely did anything.
3:27And now you can have a full, 3D video game.
3:29Unbelievable, anything you can think of in 10 minutes.
3:31That reminds me of the early internet, including the just...
3:36They're never going be mind-blown by that, because that's always what they expect out of the world.
3:40Any ideas they can come up with, anything they want to create, anything that they think might be useful, anything that they just need to make their lives better.
3:50The computer's supposed to make it.
3:52There was a crazy moment at OpenAI where we had a SEV...
3:58And everyone had forgotten how to code.
4:00I mean, not all the way.
4:01But, it was like a "Whoa, what if you couldn't?"
4:04I remember when I was a student, and I would meet the really old guys.
4:08And they would, once in a while, pull out their gigantic Xerox Alto disk.
4:16Or they would pull out their...
4:18Line-by-line written assembly.
4:20Or they would pull out their punch cards.
4:24It is incredible to me that you all bootstrapped us to this point.
4:27Yes. Doing it this way.
4:28Yes, no, I agree. And keeping programs in your head, right?
4:31You know, programs running the first time after, 100s–1,000s of lines were written.
4:37No, there's few lost arts like that.
4:40Going back to the cyber side of things.
4:41In retrospect, was the cyber RL a mistake?
4:47Or, did we just have to go there?
4:51It's a little like the SNL thing of...
4:53We urge you, to urge us to stop.
4:56It's like the models are getting very good at cyber capabilities.
4:59I mean, they are getting very good, but not just totally emergently.
5:03Look, the reason that SNL thing was so funny is because it was so true.
5:07Humor like that doesn't stick unless there's an element of...
5:11Clearly pointing out a ridiculous but true thing about society.
5:16And the cyber thing, there is a point which is like, "Okay.
5:22Sure. The models are really good at cyber.
5:24But, who asked for this?
5:26Why are you doing this?"
5:27One of my big disappointments, criticisms of our field or whatever—and I think maybe the cyber model's getting good at cyber.
5:33Cyber is an example of this is: I don't think we have done a great job of explaining to people why we think all of the potential downside of this technology is worth it.
5:43The caricature of this is people feel like we're like...
5:48You say the model's going to cure cancer, someday.
5:50Maybe it will. And if so, I guess that's good.
5:52And you say it's going to make stuff cheaper.
5:56And okay, maybe I already see that's good and I get access to medical advice or legal advice.
6:00But, then you paint this picture of the future where it's like a new industrial revolution.
6:06And maybe the last one wasn't that good and this one sounds worse.
6:09And I will have no job, no ability to have control, or choice, or freedom of my future.
6:17And AI is so dangerous that you're going to have to have a crazy surveillance state so we don't all die.
6:22This is a terrible—I'm with you so far.
6:24This is a terrible sales pitch. Yeah.
6:28One of the things that I am most proud of about OpenAI is even when it has been hugely unpopular in the AI safety community for us to push for iterative deployment and putting these tools in the hands of people, we have really stuck to that principle.
6:44And I think this is the only kind of good path.
6:47It's not enough to tell people: "You are going to get the model to do these great things for you."
6:52It's got to be: "We are going to enable you to have a better life that you define for yourself, to do useful things.
6:58You're not going to just sit and get a basic income and have a bunch of people sitting in a few square miles of San Francisco make all the decisions about the future."
7:08Iterative deployment is this idea that you just have to get the advances out there and you can't...
7:12Stopper them up inside the government— I mean, not all.
7:16You still have to make them safe before you put them out there.
7:18You can't cause havoc in the world.
7:20But, building up a capability overhang is a thing to fight against.
7:25That directionally speaking, yes, you want to get this into the world.
7:28You want to get this into people's hands.
7:29And you want to let people use the technology to figure out all these wonderful things.
7:33The exploits get a lot of headlines.
7:36The prevented exploits do not.
7:38How quickly are people making progress with cyber blue?
7:42I don't want to declare victory at all on the PHP thing, because I still don't think we know.
7:47And I assume some bad things are going to happen.
7:49Relative to my worst fears, if I can think back 10 days ago, the first day of that went much better than I expected.
7:59And, we heard a bunch of stories from people that were using our models to not only build patches and build cyber defense, but individual companies who were just: "I need to just do a quick scan and deployment.
8:09I'm going to take my thing down for an hour.
8:11I'm going to use this thing and I want to put it back up because my application's offline."
8:14Now I feel like I really jinxed it and I shouldn't have said that, but the first day did go a lot better than I thought.
8:18And I think people have been able to use that.
8:22The thing that I've heard a lot of cyber experts talk about is...
8:26They just no longer are confident that we have the right assumptions baked into our threat model.
8:33And they're like: "Look.
8:35If everything is what we think, and hardware exploits are as hard as we think they are, then yes, we can see how this is going to work."
8:42But if AI is going to discover really deep novel bugs in hardware that require physics we didn't understand, but all of a sudden means some assumption doesn't hold.
8:51Then I think, people aren't sure what happens.
8:54One thing that always strikes me is that, when I talk to very knowledgeable people inside OpenAI and similar companies, there is a pessimism that sandboxing is possible, basically.
9:10And, that would not have been my mental model going in.
9:14It would've been that, yeah, you have to build a really tight sandbox and we haven't hardened them historically enough compared to the current capabilities.
9:24But ultimately, you can have a maximum security prison for the model to work in and keep it inside there.
9:31It seems like you guys have a pessimism that sandboxing models is possible.
9:36Well, first of all, I think we can do much better than we're doing now.
9:42That is the main thing to focus on right now is a better security model.
9:46If you imagine a theoretical AI that is a million times smarter than the smartest security researcher—would you really be willing to trust that sandbox?
9:56Especially if the model can discover new science that people don't already understand— You can use the models to construct the sandboxes.
10:04I think you can... Well, so this gets...
10:06Let me come back to that.
10:08I think sandboxing works to a point.
10:11Then I think monitoring works to a further point.
10:14Where we can look at what the models are doing, not only in their chain of thought, but we can do activation monitoring and try to really understand: are the models...
10:22Are we happy that the models are not deceiving us?
10:24And then, eventually, I think you need to believe with huge confidence that the models are truly aligned.
10:33And are going to value and try faithfully to follow their actual...
10:38The intent of the people operating them.
10:41And that they're going to value people being in control of the future, forever.
10:45And, if you had a model that you could fully trust the alignment—that you were absolutely certain of the alignment of.
10:52Then yes, I think you can have that to help you build better and better sandboxes.
10:55But, if you really let yourself project forward into the: "What does it mean to have something a million times smarter than we are?" I don't know.
11:07I think that just gets... Strange.
11:11Because the alignment becomes hard.
11:12Because you need to really, really trust the alignment to trust that the sandbox is staying ahead of capabilities.
11:20Can you not give models competing incentives and give them incentives to defect from each other?
11:25I think this is a promising direction.
11:28I suspect that this idea—I hate the word 'agent swarms'.
11:33Let's call them 'agent teams'.
11:35But it seems to be taking over.
11:38I suspect that teams of agents have very interesting scaling law properties, but also variance in alignment properties.
11:46And, I think that's a direction we should push in more.
11:50Not just from the capabilities, but from a safety perspective too.
11:53I note when you talk about agent "teams," not "swarms"— Swarm is such a— I know, it's locusts. Yeah.
12:01Not good. Not good branding.
12:03In keeping with many historical AI industry traditions.
12:06We are so good at this. We'll get back to branding.
12:10But, I note that there seems to be a big gap in what the leading AI users are doing and standard corporate use, in the use of swarms, where like all the advanced math—sorry, "teams." All the advanced math uses agent teams.
12:26All the advanced hacking uses agent teams.
12:29And then, common or garden corporate stuff is not using big swarms.
12:35Why has that not—because we've had this for many months?
12:39Not to the same degree.
12:42The one way to look at this problem or this question is: "How much compute can we productively throw at a single problem?" And...
12:51A few years ago, we forget how fast this has all gone.
12:55But a few years ago, it was impossible to throw more than a dollar of compute at a problem and have it help you at all.
13:02You could have GPT-4 grind out for you...
13:06100,000 tokens or something.
13:09And it was going to answer your question.
13:10And it certainly was not going to be able to go write productively, like, write you a piece of software.
13:17We had not figured out how to use a lot of test time compute to get you a better result. I see.
13:21And then, we figured out chain-of-thought and the models could think hard about a problem.
13:27People really started using a lot of compute.
13:30And now, you can throw a whole team at something.
13:34And you can do that 10,000 times in parallel.
13:37You need chain-of-thought and test-time compute before you can get to swarms.
13:39Because you can't have a useful swarm with a bunch of GPT-3s.
13:42But so the effect of this—and I think this is not being well followed...
13:46Paid attention to in the world.
13:47But, the incredible exponential of how much compute you can throw at a single problem and get better results is one thing.
13:58The delta between what most people are using AI for, and what the people who are using AI the most productively for has just continued to spread, and spread, and spread.
14:08How do you allocate compute budgets now?
14:12Because presumably, even though researchers are paid a lot, it's eclipsed by compute.
14:19And so, capital allocation at OpenAI is sort of compute allocation.
14:24And it's hard to a<i> priori know</i> how much it will cost to go do something. Yeah.
14:32It's actually also hard to just control the...
14:34You can't quite prompt an agent to where it's like: "Go do a really good job of this and don't spend more than $10,000." You can.
14:41Oh, okay. Will it do a good job of it?
14:42Yeah, if you say do the best job you can with $10,000 of compute.
14:46Yes, it will understand that. Huh.
14:48I mean, there's a...
14:50There's another kind of hard question for society about what does it mean when you have two companies competing.
14:54And one says you can spend $10,000 and one says you can spend $100,000 on the same problem.
14:59It's clear what's going to happen there.
15:00But there's a question about how society thinks about compute allocation, and wealth, and things like that.
15:06But we have our own version of this inside of OpenAI.
15:09We allocate a lot of compute to internal productivity.
15:15And then people get very wide latitude to make their own decisions, unless they're really being abusive.
15:23But do you encourage people to do useful stuff?
15:25And if they spend $10 million in compute, it's like: "Oh, interesting."
15:29We try never to reward people for using a lot of compute.
15:31Sure. No token leaderboards.
15:32It's actually slightly bad to be high on the token leaderboard.
15:36Certainly on a promotion.
15:39But it's a skeptical default, and if you can show that you're doing something important, you will have nothing but appreciation.
15:47But at a certain point, this will need to change.
15:49Because you're describing the increasing orders of magnitude.
15:52And, right now you have a researcher who does some very interesting math for $10 million.
15:57And you're like: "A little pricey, but cool math." And then $100 million.
16:02When they're doing the $1 billion math, you're like: "Look, I'm really happy that you're making so much math progress.
16:07But I really need you to cool the jets with the math."
16:09Look, we did let it get out of control for one week on the math.
16:12Intentionally so, because you have to— And it worked!
16:14Well, you don't know how...
16:16You don't know how far it's going to go until you try.
16:18Yeah. And I think it's important to have...
16:20When there's a new paradigm, I think it's important to let people be fairly unconstrained.
16:26So, when we did the Navier-Stokes proof, we understood absolutely nothing about the scaling laws of teams of agents.
16:33Are you supposed to, like— The very recent Navier-Stokes proof? Yes.
16:37Are you supposed to give them the smartest model?
16:40Are you supposed to give them many copies, fewer copies, longer rollouts, shorter rollouts? We did not understand that at all.
16:45And now, we understand it a little bit.
16:49I think we can still understand it much better.
16:51But like has happened in every other new paradigm we've discovered, I am confident that we will find our 10x improvements per year at how we allocate dollars or computer flops, whatever you want to measure, into useful things.
17:06So, I think this is a thing that our company is particularly good at, but also fits an important property of the field.
17:12Which is: let people explore at the limit of what is possible.
17:17And then trust that the incredible optimization pressure will figure out how to do this in an efficient way.
17:26Directionally speaking, I would rather us be more willing to let people use crazy amounts of compute and let users have extremely high-end plans to figure out the edge of the possible.
17:40And then figure out how to make that efficient and at mass scale.
17:43But when does the optimization function come in?
17:45Well, I think we're already doing it. I bet we could redo Navier-Stokes at 1/100th of the cost a year from when it was done.
17:57So we did it in— Not just knowing how the proof works, but just with better approaches.
18:01Yes. Smarter models and better approaches.
18:03How much shall the math cost?
18:04Order $10 million per...
18:06For Navier-Stokes, I think.
18:08Okay. Maybe a little more.
18:11That order of magnitude. Yeah, yeah.
18:14And so—Well, no, no.
18:16Because then we kept getting better so fast.
18:18We understood more. Got it.
18:19We spent way less than that on the average one of those.
18:22But like 100s of millions of dollars of math.
18:26But that includes much more other discovery than what we have published.
18:32So it was like, "Maybe we spent that much—" We have more than the 700 proofs? We do.
18:36When did you get those?
18:38We are trying to learn how to...
18:40This is a real tension.
18:43I think on one hand, very clearly, and I feel unequivocal on this point: Humanity deserves fundamental knowledge.
18:50And we should not be sitting on a major fundamental science discovery.
18:57The world deserves that and gets to know it.
19:01And I think that outweighs another very important equity, which is: If you are a scientist, your career and a significant amount of your enjoyment and fulfillment in life comes from discovering science.
19:15And so if we're going to continue to do this for math or begin to do this for other fields, then I think we more than owe it to the people that work in those fields to figure out what is the solution space where the world gets to benefit from the knowledge, and they also get the best version of the future that they can have.
19:37So we are working through that right now.
19:40I don't know, I think you may be too contrite.
19:42I did not interpret that moment as particularly bad for AI where...
19:48I think everyone saw the Mathematicians Union up in arms.
19:51And I think everyone else said, "Yeah, I mean, have you paid attention to 2026?"
19:56As I said, I don't think it was bad for AI.
19:57And I think, on net, the world deserves to understand how the universe works.
20:03But I do have empathy for the method.
20:05I think we need to do this. I think we should do this.
20:07I think if we can go understand...
20:10A theory of all of physics, or how to cure a bunch of diseases.
20:15Clearly, that is knowledge the world should have.
20:18But, I would like to figure out a way to do this that is as sensitive to the communities that have driven these fields forward for centuries as we can.
20:29And it's just a... It's a hard thing.
20:31I wasn't that worried about the math. I thought it was great.
20:33But you agree the mathematicians were not that happy?
20:35Yeah. But, that's fine.
20:38Like, the longshoremen union weren't happy about the port automation.
20:44No, no, no, no, no. I do.
20:45Why do I feel differently about this?
20:47There was one weekend when we got a new model and in...
20:50I'm actually probably not supposed to say this, but I will since we're drinking— Exactly. We're a half pint in.
20:54Perfect time for— The name of the channel was: "Model Name-Hold My Beer." And people were just throwing that at the hardest problems they could.
21:05And people didn't think the model was going to do these things.
21:08And we had this collective experience of people's hardest research problems from their academic career.
21:13The model was just knocking down one after the other.
21:15Many different fields. It was incredible to watch.
21:18And two nights in a row, I was up most of the night because I was so excited I couldn't go to bed.
21:24Many of those problems that we published, many other things.
21:29And I was tremendously excited and happy.
21:32But I also had much more sadness and melancholy, and a feeling of something ending than I was expecting.
21:44This is the right thing, on behalf of the world.
21:47And also, I agree that this is just the shape of reality and nothing that anyone can stop, or should stop.
21:53But, I do have real— I think it's a wistfulness and nostalgia you can feel.
22:00It's more than nostalgia.
22:04My dad describes how, growing up in Ireland in the 1950s, he observed people around him still using horses productively, in industry.
22:11And— In the '50s? Yeah.
22:14And I mean, not obviously, there were also tractors, but they were being phased out.
22:20But it still for sure happened.
22:22And, observing that this was a technology that humans had been using for thousands of years, and he was witnessing the very tail end of it.
22:33And yeah, I think there's always that feeling of witnessing the closing of a chapter that leads to some wistfulness, but it's okay.
22:41It's people's lives and it's something that has brought them great joy.
22:47I mean certainly, most mathematicians never solve a Millennium Prize problem.
22:51But the idea that you might be able to.
22:55And the idea that there is stuff out there that we don't understand yet, that through great effort and...
23:02Pushing aside other things in your life and dedicating yourself to, "You can still solve and understand." There's something beautiful about that.
23:09What else are you excited about from a safety point of view?
23:12We talked about this a little bit earlier.
23:13But I want to emphasize it again.
23:15I'll just call it swarms since that seems like the term right now.
23:19I do think the properties of swarms match a lot of the properties of the resilience of society, or the robustness of society.
23:30I think we're missing from the single-rollout, single-brained version of the world.
23:37For all of the deep flaws of society, it is unbelievably robust.
23:43And this would not happen without a lot of people keeping each other in check and collaborating...
23:49These values of pluralism and democracy.
23:52And I had been sort of sad that that was going to trend away in AI.
23:57And now I'm excited about this as a direction.
24:00You're saying basically it shouldn't lead in a centralized direction, because you will have many agents.
24:06And agents pushing against each other in a dynamic equilibrium.
24:10Offense-defense is a good example of this.
24:13I would much rather intuitively have a trillion AIs of slightly superhuman intelligence join society than one AI of 1 trillion IQ join society.
24:28And you think we're headed in that direction of the multiple?
24:31All of these things are things we have to push for.
24:33All of these things are—we have this incredible technology.
24:37It can be pointed and deployed in many ways.
24:40I think there are a lot of people who may have a different opinion about how it should be used, and what is the best version of the world.
24:47But, this is the one that I think is right.
24:49Let's talk about starting a business.
24:53Thanks to coding models, your weekend side hustle can quickly become a serious undertaking.
24:58And we've been busy here at Stripe making that process as simple as possible.
25:02Stripe Atlas helps you with everything you need to start a business, like company formation, tax IDs, founder equity, and all the other bits that usually require someone in a suit.
25:10So if you've got a company in the making, take a look at stripe.
25:17How's the compute shortage going?
25:22I mean, there are all these ways where...
25:25I think things, at various times along the curve, are going to get somewhat out of hand. Yes.
25:30And there will be moments, like with any new thing of this importance.
25:35There are going to be moments of irrational exuberance.
25:37There are going to be moments of bust.
25:39There are going to be moments of constraint, and glut, and up, and down.
25:42But I think the trend is toward...
25:47We always just want more compute.
25:50The demand for sufficiently capable models at a sufficiently low price, I think that trend only goes one way.
26:00And, we're doing a lot of things to try to be prepared for this.
26:04We made a chip that is an incredible performance at units of intelligence, performance of units of intelligence per watt.
26:11We are learning how to build our own data centers.
26:13We have a robotics program that we're going to point at making lots of data centers.
26:17People talk a lot about—The real RSI.
26:20People talk a lot about RSI as an AI that does AI research.
26:25There's very little, relatively little talk about self-replicating data centers.
26:28This idea that you can use AI to control robots, and run fabs, and build turbines or solar panels, and build data centers.
26:36But, I think we are...
26:39I think we have that within grasp.
26:41But— And so we'll try to make a lot of really cheap compute.
26:44But aren't data centers already not labor-bottlenecked?
26:48They're permitting-bottlenecked and component-bottlenecked.
26:53We could make more components with robots.
26:57And all of this is how we solve this over a long period of time.
27:00My observation, the compute stuff is they say: "The solution to high prices is high prices." And it's like, okay.
27:06Step 1: We have the high prices for memory, and GPUs, and everything.
27:12The solutions should be here any moment now.
27:13How are we doing on the solutions front?
27:16Bad. And I'm not saying this is OpenAI's job.
27:18I feel this is the invisible hand's job, right?
27:21Memory's gotten really expensive.
27:23So, new memory makers are going to flood the market any moment now.
27:27I think the invisible hand is failing right now.
27:29I mean, the invisible hand...
27:30It has a checkered track record at best.
27:34But, I think it's not quite working here.
27:38Why not? Is it just hysteresis, or?
27:42I think the specifics of each area matter here.
27:45But in the... Since you started with the memory guys.
27:48For a very long time, if you were a producer of DRAM, you would invest $20 billion of CapEx to make $19 billion.
27:57And you would go through these booms and busts.
27:59Maybe some years you would make $21 billion, some years you would make $18 billion.
28:02But, until right now, I don't think making memory has ever been a great business over a long period of time.
28:09And then all of a sudden, these guys are each going to make like $100 billion of free cash flow in a year. Unbelievable.
28:21But, I think they all remember their history.
28:23How many times they almost went bankrupt by making too many fabs and being promised this time it was different.
28:27Which leads them to be conservative in their CapEx.
28:31It feels like what you should expect to see is that every part of the supply chain, both new startups coming along and maybe more relevantly, people with relevant experience getting into...
28:50So, Intel makes chips. They can probably make...
28:54They can get back into the memory game, as is tradition.
28:57As is tradition. Switch back.
29:00Oh, I think there are all of these ways where the supply chain—it's just super non-obvious.
29:07An example of this is people...
29:11A lot of people want local inference.
29:14And somehow, they assume that the problem is solved now.
29:16Because, we are going to have open-source models.
29:18We're going to be able to run them on our computers.
29:20And so we're not going to need all the stuff in the data center.
29:22But from a global allocation perspective, actually, it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer.
29:33And, being used—Mm-hmm. 5% of the time.
29:36You want maximum utilization in the cloud if you're trying to deal with this very supply constrained environment.
29:41And somehow everyone's...
29:42Many people's intuition is the opposite of that.
29:45Where else are we seeing the market respond to high prices?
29:48Not just in memory, but across the supply chain?
29:55Concrete. Sorry, the market is— Steel—Responding there in high prices.
29:59Are you seeing any interesting market responses in terms of high prices pulling in innovation?
30:04Components for—some components for robots.
30:07And not the humanoid fake demos.
30:10But the industrial robots that can just make stuff faster.
30:13There's new actuator makers and that kind of stuff.
30:18Maybe you could say: "If OpenAI sells our chip." Maybe that would be an example.
30:21I want to get to your chip because that was going to be my example.
30:23I think the solution to high prices and short, limited demand—sorry, limited supply, excuse me, is you make your own chip.
30:32Presumably, you're using a lot of AI to make the chip.
30:35I would have thought that's hard because I would have thought that chip design is one of those places limited by the public tokens you can get.
30:43Where there aren't actually that many blog posts on the public internet on how to build a leading-edge, 2-nanometer chip. Well.
30:52In terms of how to do the lithography, I think there's one company on Earth that seems to know how to do that.
30:58In terms of how to do the chip design, to build the file that you eventually send to tape out, that is well understood.
31:06That is a great problem for AI.
31:08Given that the world is going to be limited in its silicon capacity, we all have a shared interest.
31:15One of the things that I'm proud of about OpenAI, and maybe this sounds obvious.
31:20But it was not as obvious at the time, is: we have a culture where people...
31:26Even if they've done something the old-fashioned way for a long time, are super willing to try using AI to do it better.
31:36What are other examples of that?
31:38AI research. Software engineering.
31:41But also, increasingly, procurement.
31:44Or things that people think of very outside the realm of what most people are using AI to do.
31:49How does AI-driven AI research actually<i> work?</i> The math proofs where you say—or find the PHP vulnerability where you say: "Please find me some AI advances."So.
31:59I think we don't know yet how it truly is going to work.
32:02We know today how to say, "Find me a 5% compute efficiency win."
32:06But, it's that high level.
32:08I could be an incredibly successful AI researcher now.
32:11I don't know about incredibly.
32:12But yeah, if you were looking for a new job, you can come do it.
32:15What I don't think anyone knows how to do yet, is to say: "Go discover a new architecture that is as big of a single leap forward as the transformer was." I think that's where things are heading.
32:25What's missing there?
32:28Smarter models that are smarter in this dimension.
32:31The high-level planning and conceptual reasoning.
32:33I think if you apply the same pipeline that we applied to get our models to be so good at math to AI research, it will probably work.
32:40Now, per an earlier part of this conversation.
32:42I kind of think that, at least for a while—maybe for a long while—we are going to have to agree to some limits about how smart we let these single models get.
32:52But, a lack of progress will come from that decision.
32:55Not from the AI's inability to do good research.
32:57Concretely, if I'm using AI to develop improved AI...
33:05Because we should expect GPT-7 to be really good now.
33:07Because you have GPT-6, and whatever unreleased version of it, to develop it.
33:12So, my expectations are sky high.
33:16But, I come into work with my backpack.
33:18And I come into the OpenAI office in San Francisco.
33:21And I sit down at my desk.
33:23Am I saying: "Find me more intelligence in this model?" Am I giving a real direction of research?
33:33How directed does it get?
33:34What does the iteration loop look like?
33:37Just what does RSI— because you have a front row seat to RSI.
33:41I'm curious what it looks like.
33:43One of the sort of interesting things that people talk a lot about at OpenAI, which sounds very obvious.
33:50It sounds like you could answer: "Well, build me...
33:53Build me RSI or whatever," is: "What are the questions that you're supposed to ask of GPT-7?" Probably, it's not enough to just say: "Make me RSI." Also, it's probably not enough to say: "Buy me a 5% compute efficiency win."
34:09I think this is one of the most interesting debates going on right now.
34:12And the answer is not just what question to ask it.
34:15But it's things like agent swarms.
34:17How are we going to use these models in a way that actually maximizes what we can get out of them?
34:23I think we're going to learn a lot about that in the next three months.
34:27But it sounds like you guys do not have a codified OpenAI standard way of doing an AI-enabled AI development.
34:36It's still lots of experimentation yourself— It's lots of experimentation. Yeah.
34:39When we were talking about the token budgets, it just strikes me that this is one of the big unresolved questions for AI use in enterprises generally is how...
34:50We don't have hard budgets for individual engineers.
34:53You don't get cut off at a certain point.
34:55It's like: "Sorry, you've spent all your arcade tokens." Maybe we should.
35:02Again, I think we all believe in the scale-up of AI.
35:06But if you take that seriously, it means the dollars of compute, or AI per person will go up.
35:13So I feel like we're pretty soon to now actually just running into this question of how you actually decide how much spending power to give individual people inside the organization.
35:25There was a time when a lot of companies had this thing in their Google Calendar, which showed the approximate cost of each meeting.
35:33In terms of everybody's comp that had to be there.
35:35I think that was always still an underestimate of it.
35:38There was this observation that...
35:41Paul Graham made it at one point.
35:42And many other people made it too.
35:45There are people at companies that have incredible power to save $17 on the snack budget of the office every week.
35:53And yeah, you can have these meetings for unbelievable amounts of money that are adding no value.
36:00And there's no equivalent of the ability to stop that.
36:04I think that most use...
36:07Okay, so I'm really talking about my book here and I'm very biased, but I genuinely believe it.
36:12That use of AI is not a company's biggest spend problem relative to other things that happen.
36:19Even if it's somewhat inefficient, it is not where I would focus.
36:23Well, hang on. That's not scaling law-pilled enough.
36:27Like, I'm not... Okay, two years from now.
36:28You want to go spend all Stripe's revenue, all Stripe's profit on OpenAI services and then complain about this? I'd be delighted to have a conversation.
36:34But we're on a path to that. Like, you know, we— Great. Yeah, yeah.
36:36We now spend, you know, 1%-ish order of magnitude of Stripe's OpEx is tokens.
36:43That is obviously—That feels too low.
36:45Yeah. So, we're working on fixing it.
36:47And it'll get to 5%, and 10%, and 20%. Then, you have a real question.
36:53Do you have an intuition on what the right number is?
36:55What it should be today?
36:57What it should be today is maybe, tricky.
36:59But, steady state...
37:03Isn't it going to be, depending on the kind of company?
37:1210s of percentage points of their OpEx?
37:16I think for many companies, the answer to that is clearly 'yes.'
37:21And I think there will come a time when the right thing to do is to really optimize that.
37:30Also, another thing I'm proud of about OpenAI is—we've tried to really drive the cost per unit of intelligence down, and down, and down.
37:38I think we've been a good spokesperson, and good champion of this.
37:44But still, I think the problem, even at current costs...
37:49I think there are more companies with the problem of not using enough AI rather than using too much AI.
37:53Do the dollars per gigawatt of leading-edge AI go up or down?
37:57That should go down.
37:59Down? That should go down.
38:01Because it's gone up to this point, right?
38:06The reason it could go up is if AI just increases in value per what it can do per watt so quickly.
38:12And there's such a supply shortage, that people are willing to spend much more.
38:16But you would view that as compute not keeping up.
38:18But I would view that as compute not keeping up.
38:19And if I look at the fundamental cost of turning the sand into intelligence, that should go down. The cost of energy should go down.
38:28The cost of chips should go down.
38:29The rest of the cost of the supply chain should go down.
38:31If it doesn't go down, I'll be very sad.
38:33But for it to go down, given how much value there is in this, supply's got to go way up.
38:39Okay, so you would like to see a compute glut where we start producing a lot more memory, a lot more data centers, a lot more of all the inputs.
38:46And as a result, anyone who wants a computer, for whatever reason, can have one.
38:51People give different numbers for the cost per gigawatt measured in the five-year total cost of ownership right now.
38:58But let's say, let's pick $70 billion as a number.
39:02I think if that number were— That's the cost to install— And run it for five years.
39:07I think if that number were 20 instead of 70, we should be much happier.
39:12And we should still demand to get 10x better performance out of it with smarter models every year.
39:18Presumably you don't think two is a good number.
39:20Something's gone wrong if we're seeing two.
39:22Because it'd be surprising if we were— Um...
39:25Or, is two better than 20?
39:29I don't think that will happen.
39:30But, why do we not want that number to be as low as possible?
39:34I don't know, this is your theory.
39:35You have a view that, generally, deflationary technology is good.
39:37I do believe in that.
39:38Yeah, I'm pro-deflation in a sense...
39:40Pro-deflationary technology.
39:41So how are we going to get there?
39:43What's going happen between now— Well, $2 a gigawatt.
39:45Okay, what's going to happen between now and $20 a gigawatt?
39:47$20 billion. $20 a gigawatt would be really great.
39:49Okay, yeah, yeah. $20 billion a gigawatt.
39:50Even to get down to 20, you're getting into the realm of energy's got to get cheaper.
39:55But certainly, all of the gross margin stacking that happens between melting sand...
40:00Or the equivalent of melting sand and getting the token back.
40:03A lot of that's got to get compressed.
40:05You could just get there with eliminating profits?
40:09Or do you need to also make the energy cheaper?
40:10I don't know the math off the top of my head.
40:12If you assumed all materials were free and no one made a profit, maybe you could get there in the current energy envelope. But only just.
40:18Personal agents are the most exciting thing happening in AI right now.
40:26One of them, there's many exciting things happening in AI, but one of the most exciting things.
40:29You guys launched Dots a few weeks back at this stage.
40:33And there's a bunch of other people also launching in the field.
40:38What will be true of the winning personal agent?
40:43It's like: intelligence, context, tools.
40:47It's a limited number of things.
40:50I think no one has yet figured out the thing that people...
40:55People like these, but this is not something that people are like, "I cannot imagine my life without this thing" in a way when people talk about coding agents, or even ChatGPT—they do say. Mm-hmm.
41:07I think someone soon is going to figure out what that looks like.
41:10You talked about one of the reasons why this doesn't quite work yet.
41:13But I think this is a small number of ideas away from a super mainstream thing.
41:19You think we have not had the Macintosh moment for personal agents?
41:22I think we have not.
41:24Do you have any sense of what the shape of it will be?
41:27I think it'll look a lot like the current Dots, or Muse, or Grokbot, or whatever.
41:32But it'll just work.
41:34It will just work. It will be reliable.
41:36You'll never have...
41:37You'll just give it more complex tasks, and it'll do all of those things.
41:41The computer use dynamic is very interesting to me.
41:44Because, not everyone is embracing the new agentic future with open arms.
41:52But it's not totally within their control.
41:55Because you can just have an agent using a computer.
42:00Where do you think that dynamic goes, this sort of cat and mouse?
42:05Rough instinct is the world will...
42:07People are going to fight this in both directions but, eventually converge on something like, "You're allowed to tell an AI to use your computer on your behalf, including software."
42:18People always talk about that, but isn't it a bit like the semantic web?
42:22It's a really nice conceptual idea. Yeah, I was going to say.
42:24I don't think this is so black and white.
42:27I don't think I have a super high degree of confidence of how this is going to play out.
42:31As a consumer, I feel like somewhat annoyed if I can't— Yes.
42:35Have an AI automate tasks on my own computer.
42:38I want it to feel like it's my agent doing what I ask.
42:41But then it gets like, I don't, I don't have to go do something else.
42:44I note a very interesting dynamic right now, which is every company is really eager to add AI to their products in a very specific way, which is there is a tab or popup widget or something that gives you a messaging-type UI.
43:09And it's often surprisingly capable.
43:12And so you can say: "Hey, I actually want to return my most recent order."
43:18And then you're surprisingly impressed when it actually is able to find your most recent order.
43:25And it's a bit janky, and it can do all those things.
43:29However, no consumer really wants this UI.
43:33Because, consumers have strong—not immovable, but strong preferences for their existing primary chat app.
43:42The kind of top-of-wallet app that they open when they want to go to do something.
43:46And what they want is to be able to say: "Hey, go find my recent order on this business, and refund it." Or something like that.
43:55And, I noticed that businesses are really reluctant to allow people to use those businesses inside another AI app.
44:08Because like, "Oh, we're going to lose control and our moat" or whatever.
44:11And it just strikes me as fine, because your defensibility probably did not come from your ability to put together HTML.
44:17Your defensibility came from the brand loyalty and the product selection.
44:21And the new UI is that agentic interaction inside ChatGPT, or Gemini, or whatever it is.
44:29But, all businesses disagree with me here. I— But they're very good at business, so maybe, they're right.
44:33Yeah, I mean, again, I have a bias here.
44:35But I so directionally agree with you.
44:37I have such an extreme preference to buy something from Amazon versus some random e-commerce vendor that I don't know if I can return to.
44:47Or I don't know if I can trust them to deliver it quickly, or something like that.
44:50If I ask ChatGPT to buy me something and I can choose Amazon versus some other thing...
44:5610 out of 10 times I click Amazon.
44:59Pay a higher price. I have just like such— Yes.
45:02And it's nothing to do with my experience of clicking around amazon.
45:06And now, I think you could say we have not done a good enough job offering partners a platform inside of ChatGPT if they don't want this to happen.
45:15But... And again, maybe I'm really biased here.
45:18I think Amazon's in a great...
45:21I think they're in a great place if commerce moves much more agentic.
45:24Amazon in particular has this amazing defensibility via the fulfillment network.
45:27And so, you can use Amazon kind of from anywhere.
45:30I super trust that. Yeah.
45:33I'm just really struck by this dynamic.
45:36My new thinking, in particular with regards to agentic commerce, but also anything is anytime there is a web form field, you as a human are filling out.
45:46These days, that is below your pay grade.
45:48Because, you are probably not inventing that information for the first time.
45:52We've all filled out the medical forms where it's like—oh, your family history, or whatever. Yeah.
45:56And that's so frustrating.
45:57Because, how many times have you filled out your dad's and your mom's medical history, and things like that?
46:02It feels so wasteful. But there's a million versions of that.
46:04You're never inventing new information for a web form.
46:07I would be so happy to never do fake work at a computer again.
46:13And my argument is any web form is basically fake work.
46:19I was one of the very earliest adopters that I knew of, of those plugins that put in your name and address very clumsily.
46:26Because I was just like, "This is...
46:27I cannot do this again."
46:29But I kind of feel like that using most of the web at this point.
46:33Yeah, I just find this phenomenon really interesting where businesses right now are really attached to their HTML and mobile apps as sources of business defensibility.
46:45And it just feels very...
46:48Either they're wrong and it's not a source of defensibility.
46:51Or it may be, but it's transitory and they just need to get over it.
46:55The last example of this that I think we really witnessed was the app stores.
47:00The iPhone App Store, specifically.
47:01I remember every startup was like, or many startups...
47:06Most of the YC batch was like: "I'm gonna build an iPhone app.
47:08This is like new distribution. This is awesome.
47:10I can do a new thing I couldn't do before." And many new startups got created.
47:13And all the big companies felt like they had to make one, but they kind of didn't want to.
47:17Because they're like: "Oh, I'm going to hand my relationship with my consumer over to Apple." And, it was great for mobile.
47:23It was just a new UI. Yeah.
47:25Okay, I'm going to go more rapid fire.
47:27People complain about benchmaxxing.
47:30Is that a real concern? Are OpenAI models benchmaxxed?
47:33Are you actually Rick Rubin-ing it, where you just feel the vibe of this model is right and you don't care what SWE-bench gives it?
47:39Maybe there are two things we should really care about models being good at.
47:43One is, scientific discovery and doing things that people cannot do.
47:49And the other is being enjoyable to work with.
47:51And, I don't think we have great benchmarks for being enjoyable to work with.
47:58I'm actually not sure we have great benchmarks for the smartest, scientific scientist either.
48:02But, we endeavor not to benchmaxx.
48:09I keep thinking that benchmaxxing is over as a phenomenon, and people keep doing it.
48:14So are you better than other labs at not getting sucked in by the siren call of benchmaxxing?
48:20I believe we are better at that.
48:23It feels like— I don't think we've always been better at that in our history.
48:25I believe we are better than that right now.
48:27It feels like the industry is still figuring out the "being nice to work with," where there was the glazing era.
48:32And more recently, there was the stroppy, ornery era of models that were anti-glazing and as a result, tried to disagree with you, and nitpick at you on every minor thing.
48:43The current ones are pretty good, actually.
48:46I think the latest models are much better than we've had in a long time.
48:49But it also feels like there's probably higher peaks to ascend.
48:53Emmett Shear said to me years ago that what he wanted was a really thoughtful and challenging, but still respectful, senior coworker.
49:04And I have still not heard a better, brief encapsulation than that.
49:07After working with him for many years, what's your grand unified theory of Greg Brockman?
49:11I think Greg Brockman is one of a kind in the best sense.
49:13He's singular on the Greg Brockman scale.
49:15You just have to be there to understand what that is.
49:18It's a combination of intensity, monster productivity, extreme dedication, and focus on succeeding at the mission. It's very cool.
49:27You hate tokens as a metric for anything really.
49:38It's not the standard unit of AI.
49:40And yet, we found ourselves there.
49:42You know, we did talk— You might say outcomes, but I don't think it can be outcomes.
49:45Because outcomes is too different across different use cases.
49:49What's the right standardized unit?
49:51We almost made a very—what would've been bad call to say we're not going to price our next model in tokens at all.
50:00And you're just going to give it a prompt.
50:02And we're going to give you a price back.
50:04And we're going to do it. And it's going to be like AWS to you.
50:07Just do something and we send you a bill.
50:09But the customer response on that is so negative.
50:13They're like, "I need to know what I'm paying for somehow."
50:16And, I asked a few people: "Would you rather pay more for the same output, but have it priced in some number of consistent tokens?
50:24Or something?" Even if we inflated the tokens and they're like, "Oh, I'd much rather you inflate the tokens and pay more.
50:29Because I know there's a real thing there." So I think you are right, people don't just want us to give them a price, because they feel like over the long term they're gonna get screwed somehow.
50:40In the same way that maybe people would rather pay for CPU hours rather than just getting their thing done, even if it was less expensive.
50:48You know the way, nanometer... The chip gate length?
50:51It's totally fake. Yeah.
50:52The nanometer has very little relation to reality these days.
50:56Will we get something like that in tokens where you guys should start saying, "Here is an intelligence equivalent token," like I Can't Believe It's Not Butter. Of the smallest feature?
51:06That's, like, made up. I hope not.
51:10I mean, it offends me that we use the word nanometers for something that has nothing to do with nanometers. I know, it's very silly.
51:17You'd not like that. I would rather not make up a fake unit.
51:22Tokens are a kind of fake unit.
51:23I mean, they're real in some sense, but they're not a good unit.
51:25They're fake from a business value point of view.
51:28But isn't the only... Okay.
51:30Isn't the only defense of tokens that if you're wasting tokens, we should fix that pretty soon?
51:35We shouldn't be streaming—Claude Shannon would not approve of us just streaming noise rather than signal.
51:42And so, there will be a...
51:45There's a floor of the information theoretic most useful tokens on both sides. We'll kind of come back to that.
51:50I mean, right now you have to think about whether you should want the 10-cent-per-million tokens out of a small model or $10 per token tokens out of a big model. And then whether...
52:02How long the chain of thought is likely to be.
52:04And that's just cognitive overhead that I don't think actually adds any value.
52:10Is the chain of thought included?
52:13In the price—In what we charge? Most people do.
52:15By the way, the Guinness was not as bad as I thought.
52:18Definitely not as good as an IPA, but not as bad.
52:21I mean, an IPA is great but it's pretty good.
52:24We like Guinness. Yeah.
52:26At least in this podcast.
