0:00As a business leader, there's a tragedy of the commons. If you want to stop, if you want to go slower, why don't you go slower? Like, I'm competing. I want to win.
0:06There's almost two camps. There's one camp which believes that this actually is an engineering problem, and there's others which actually believe you have to slow it down.
0:14Humans don't respond fast enough to the attacks that are happening. You need to automate all of those. And most organizations are actually not close to doing that. Is RSI and recursive self-improvement that the labs are doing leading us there? That's the big question. something Elon said, this is some elaborate 4D chess because on the one hand you're saying all of humanity will die. On the other hand, you're saying, "Hey, what do you want for your IPO allocation?"
0:34For the first time ever, a company at scale last week said that they're moving from the frontier models to GLM. Do you think that that's a trend or do you think that's just like a one-off anecdote?
0:44Um, thank you for being here, Ellie.
0:49Super excited. So, we obviously want to get to data bricks, but there is a broader conversation going on right now about AI and Daario's weighed in, Yakob's weighed in, Elon's weighed in, but we want to hear what Ali Goodsy thinks in terms of, you know, if you called the topic broadly speaking pacing the frontier, etc. Um, what is your strongest agreement with what's being out there? Where do you disagree? And maybe where is there nuance that's not being captured?
1:18Yeah, happy to cover it. and me and Martin argue a lot. So, um I'm sure that's not gonna take long.
1:23We'll try and we'll try and rain it in this time.
1:26Try to stay calm. But, uh well, I I do think first and foremost that there uh maybe we agree on this that um uh leaders have responsibility to not freak people out unnecessarily unless there's really really good reason. And I think you know there's always different people in society that are at different places you know in their mind space. So, you know, talking about these kind of existential risks and, you know, uh scenarios where all of humanity is going to be wiped out, I think, uh, um, is irresponsible like it can tip a lot of
1:55people over and it can cause a lot of like mental health issues unless you have something that's going to wipe people out.
2:00Yeah, as I said, yeah, if if there is a actual reason for it, then, you know, that's a different story. But I think that right now the existential risk is close to zero. Um, so why freak everybody out? It's not actually needed. Uh, there are risks. We'll get into it. That's probably where we disagree. Um, but first and foremost, I think that leaders should not freak everyone out.
2:22And I mean, you know, if there's like technical nuances in how we're doing AI research and so on. Well, researchers can discuss that. You don't need to every time go on TV and or blast on Twitter to millions of people that, hey, you know, I think there's like this percentage 10% risk that all humanity is going to be wiped out. I don't think that's like helpful for a lot of people.
2:40actually I think causes a lot of harm for a lot of folks who get stressed out and actually are not in the nuances of all of this stuff and what it means. So that I don't think we should do. Uh I don't think it's fruitful. It doesn't really help anyone.
2:52I I mean I think this is very very true for the general public.
2:57Like my sister who's great who's a school teacher in rural Arizona on Sunday texted me and said and said should I prepare the cabin for you? She's kind of a prepper anyways, but should I prepare the cabin for the AI apocalypse? You know, I've got water set up. Like, when are you showing up? I'm like, hold on.
3:15Like, we're not there yet. So, clearly this has kind of spilled over the populace, which I agree is unnecessary and has blowback. I think there's a second one which is um I don't know if you saw like walking in here, I was checking X and Elizabeth Warren just talked about um pausing all of AI development. That of course is on the coattails of Bernie who is also working with Bannon like Steve Bannon like so now so I in addition to like you know just scaring
3:43people the federal complex is now spinning up and I think that could be actually quite contrary to the actual goals of the message and so there's more than just you know I think public hysteria at stake here. Yeah, there's a lot of politics going on, but I'm like in all of these groups and you know I see both sides. There's there's heavy politics happening on both sides, we should say like right.
4:06Oh yeah, this is happening both sides.
4:09No, this is this is no this is a this is a I think both parties that do not include Trump himself agree that um that AI should be constrained at some level.
4:20I'm talking about the other side of this argument as well. Let me give you an example.
4:22Even even Greg Abbott, right? Even Greg Abbott was like, you know, you can't have data centers in Texas.
4:27Well, I'm not talking about politicians.
4:28I'm talking about there's politics going on on both sides, right? There's politics on the business side. People who want to see great IPOs and they want to get returns on their investments and they're like, don't mess up my IPO and they want to get like, hey, can everybody just shut up so that we can get our money back. Uh, so there's that.
4:45and they're, you know, they have resources and they're uh using them and, you know, so there's politics on that side and those are like not they're not sitting quietly and not doing anything and they can pull strings and they have connections. On the other side, there's all the people that like, okay, how do we weaponize this? This is awesome. This guy tweeted this, you know, let's like let's weaponize this one. Let's plant this. If I, you know, let's pump these, you know, threads.
5:05Let's let's talk to the specific leverage point that everybody is using because I actually think that this is like a classic case of a PR misstep and it's not just the doomy gloomy type stuff. So, here's the PR mess. I think which is um like it is not unusual for industries to try and regulate themselves. It's just not right. And I think saying like security and safety is important. It is with every techie puck and we want to have some oversight. That was very very sensible. The problem is is just couched in this notion of pacing and there's there's a number of issues
5:33with pacing. First off, it's orthogonal to safety and security. Like you can slowly build a weapon. That's not different than building a weapon. people don't feel it's it's it's genuine um because like these companies have been at a dead run if they're still buying more compute to be even faster.
5:50No, no. I mean, no. I mean, like they just haven't done it. They haven't done it historically, but also it kind of it kind of feels like this kind of almost milk toast capitulation to the pause people. So, you're like, well, you say pause, well, I say pacing, which is almost like pause, but it's not like pause. So, like they they cho they chose this kind of like flag to follow around pacing. But if you actually read, did you read the the the document that D wrote? It's a totally sensible doc.
6:15Yeah, I read it. Yeah.
6:16It just has nothing to do with pacing, right? And so I I honestly No, he does mention it. Looks I I kind of a little bit disagree. I look, there's a tragedy of the comments.
6:23There's this like, hey, if you want to stop, if you want to go slower, why don't you go slower? Why do you write articles? There's a lot of people making that argument. But no, I mean, as a business leader, I understand that there's a tragedy of the comments. Like, I'm competing. I want to win, you know, and you're also the market equilibrium, which suggests that pacing is probably impractical anyways. Yeah. So I'm just saying that the you know so it makes kind of sense for people to say hey if you guys don't stop this this strategy of the comments is going to continue.
6:46I'm not going to stop racing because you know there's IPOs at stake. There is a competition at stake. There's also some animosity between the people. So like I'm not going to stop unilaterally. I'll be a sucker. You know why don't you stop first? You know so then they're saying hey can can you come in and stop us? Um but you know I think that uh you could also make the argument that if you look at the hugging face openai incident that uh by the way I think these companies are great and I think they are probably investing a lot of resources but it's very clear from if you read what happened
7:13is that uh they weren't monitoring every token coming out and having it you know they were just like running these RL experiments and then after the fact coming in and checking what happened so they should have paced they should have been much slower in that particular incident right I I just don't want I don't want just quibble on se on on on on syntax but like words matter with PR right so let's take the hugging face incident when I read that you know you know what my reaction was is was not oh open AI
7:41should pace it's like dude [ __ ] secure your thing right like do security controls like we always have done like it is pacing though it is pacing it's not pacing like in the history of the internet we had all of these things like let's pace the growth of the internet let's do security let's do control let's do like what I mean you could you should do that but it is pacing in the sense that look I face this all the time. I have a legal department at data bricks. I have a security department at data bricks and you know they're always like hey slow everything down for everything not AI like literally every little thing like oh you're going to go on a podcast well what's the script for
8:11it what are you gonna say and you know let's review that and you know what's the legal you cannot say this you can say that you can say that you know everything you say have to materially true you cannot he's pacing people in this room with us right now so you know so you're running a RL experiment you're training the next model should the security team be there and look at like run all their monitors and look at everything I mean like millions of hours of GPU hours of tokens were produced and these agents were running you know a mock in the sandboxes it would have slowed them down significantly if we had security team sit there and look at all the stuff I agree now and they're saying that hey like you
8:40know if we do that it'll slow us down and I'm not sure the other side is doing that so can you guys come in and slow us down like just tell us like put some guardrails around us we'll happily then follow the rules and do the secure thing um otherwise it doesn't make sense because we'll get our I just I just think like nuance second order words don't work when like people are really afraid you're like I'm going to pace and therefore or things like these don't happen. I literally think we should have just been like swifty safety and security is paramount. We're going to put in these controls. Like that's the important thing. And I do think that
9:09nuance actually got lost if you look at what Zuck said. Do you agree with I thought Zuck did like hey we're going to pace ourselves. We're going to put in SEC like the reason we released this later is because of security.
9:19Well listen what I thought was so great about Zuck is like he was very focused on like like security um safety and self-regulation. Daario Daario's first five words or whatever are like we need to pace the frontier right it just puts you in a very different mindset than what he could have said is we need to secure the frontier fine we need safety the front I mean there like at some level I think they were trying to optimize both for the doomers which cause for pause and for
9:48politicians and they kind of didn't satisfy either but those people are actually freaking out inside the labs and there are a lot of safety people that are freaking out genuinely by the way and not all of them are EA people and so on People are like, "Hey, they're surprised." Right.
10:01Right. But here's the thing is like using the worst pace doesn't help either of them. I think I think it's like literally you're like you're like you're trying to find this like he he pace is like the uncanny valley of making the doomer people unhappy and the policy people unhappy because the doomer people are like that's not a pause. This is pacing. Yeah.
10:18And you know everybody else is like well like this is you know this isn't real. You're not going to do it anyways and and you're not focused on security. So so again independent of what we should do which we should talk about. I just think that the way it was presented was just bad and it just didn't work and that's why we're having the blowback.
10:33These guys are, you know, uh they're not trained uh, you know, PR people, you know, and yes, some of the stuff I agree I agree with. I mean, I agree with trained PR people.
10:42I agree with the core premise that we shouldn't freak the public out. I think that existential risk right now is close to zero. um you know uh but let's talk about the core thing which is um the the fact that you know anyone who's doing big reinforcement learning runs and they're giving it a reward function. So unleashing you know saying hey here's like a you know 10,000 agents and here's $100 million let's put them in parallel and let them run on a gigantic cluster for a month or two try to solve anything
11:11and it doesn't need to be a security thing. It could be like do anything you know solve this math puzzle really bad things can happen really bad things meaning things get hacked and it has you know cyber is the primary one right that is real right I think of this making it hey this is an existential risk and so on which I think was a mistake I think it's not good to scare the public that way um I think it's become something that everyone not just your sister everybody around the planet is like now talking about I've had all kinds of people that never care about this stuff and they find this extremely boring uh ping me and say what
11:40do you really actually think about this is really important in my account. Now I'm starting to worry about it. Uh so then it becomes a political issue and we have elections here coming up but there's elections all around the world.
11:50Uh so you're going to see they're not going to sit still in other parts of the world either. Uh but I think that's our responsibility to talk about this uh in a balanced way and actually expose the risks. I think that super intelligence that idea from that book is very very far away. I don't see any evidence that we're actually marching towards that or that's going to happen. Apparently some people Yeah. Apparently some people at the labs are freaked out that maybe there's progress towards that and I think it comes from RSI recursive self-improvement the model is improving themselves. Uh I would love to
12:18understand how much what have they seen something we don't know. Uh there's you know kind of four criterias if there if those four things are happening I would love to understand them. One is our models uh the ne if if we end up in a situation where following four conditions are happening which is the next model require less resources less GPUs to train and you know super linearly not just like tiny little bit the next model uh you know takes less time to train as well. So the second condition uh third accuracy of the model
12:47the intelligence is increasing and fourth we can do the former three again and again and again in a you know it's not just all all of those at the same time right all at the same time not just any of them yeah all four if all four are happening uh then you can imagine in a way where you can you know cuz any of them does not happen like for instance if resources is constant then that's okay because we're going to run out of hardware so then it'll pace itself like we will not have enough hardware to do that not enough GPUs right uh time the same. So it needs to be that you end up
13:15in this situation. So if it's if you just mean that the software is writing itself, uh we're already there today like 90 some percent of the software in data bricks is written by AI. Does it matter if the last few% is also written by AI? No, it doesn't matter really that much. Uh but if you're getting these four conditions then you might get a speed up where the next model let's say takes half amount of time and half the resources and it is more intelligent and you keep doing that you know uh then you might end up in a situation where I don't know by the way I don't even know
13:44if that necessarily leads you to super intelligence per still converge yeah but it could so then that would be more risky so that it would be nice if uh they can share all that data and we can shine some light and transparency on that I actually think it's a great breakdown that you have I don't think anyone is using that as a definition actually right I think people I think there There's a little bit of people freaking out about like oh my god emergent behavior now it's creating itself and so on but I think like as I said a lot of people their definition is just hey if I'm not even coding anymore and it's coding itself right uh but I think they're conflating hey
14:13what's my value and is it scary for me versus hey that then means we'll get that super intelligence that 2014 theoretically was uh hypothesized bystrom um well your point in compute though is a really good one that's missed in a I think in a lot of arguments on RSI right Because as far as we can tell, the minimum threshold for compute needed to train a good model just keeps going up.
14:35Like it was 100 million billion now it's probably like 5 billion. Um and so that you train a model now to train like a frontier model. Right.
14:44Right. Right. Exactly. Versus billion billion billion billion. Yeah.
14:47Versus very expensive.
14:49Yeah. Frontier is very expensive to to replicate the frontier 6 months later is about 120th the cost. No, I think Sarah has a great point which is it's this is a good argument against this whole thing which is that the next model first of all there's only one or two such runs a year that each of these labs do and they take it's the opposite of the four criterias that I mentioned right which is it's going to take more resources more humans involved and it's even more brittle and they have to build out the data centers I mean like the labs are not necessarily doing that but others have to build the data centers and they have to be gigantic and they
15:19have to get the GPUs and they have to get the networking right they have to do the engineering to make sure that they can tolerate because you know every order of magnitude more GPUs you cram in there, you have to now worry about errors that before you didn't have to worry about. So you have to increase robustness of the So it's like a very brittle process and if it fails, you've squandered so much money. So they're like very very careful with that run and there's been multiple runs that have been botched.
15:39So it's it's the opposite of that that hey the next model is faster, cheaper, smarter and recursive improve. It's the opposite. It's like it's taking longer and it's more brittle and it's more people and it's harder to pull off. So, um I do think that that is true with respect to RSI, with respect to actually cyber risks and things getting hacked. We need to take it super seriously. Yeah.
16:01So, I'm I'm going to have like another like miss here like I actually love your four criteria. I was like literally just waiting to argue with it, but I actually think it's this is very good. Uh so, I'm let me give you like um like a blackbox what when like when you're dealing with these like dynamic adaptive systems like what are you going to believe? Are you going to believe like the numbers or your lying eyes? Right? Right. So, I think you kind of have to go to the numbers on these ones. So, like what are the numbers to look at? I really think you should just basically and maybe going public is the right way to do it.
16:27Like like if if these companies continue to grow, reduce the number of people and the number of amount of money that goes into them, um then I would say something is is definitely happening here. Like I do think that like you can actually blackbox this and take a look. But none of those indicate like they're hiring like crazy. They're like crazy.
16:45That's not fair because you know companies are not necessarily efficient, right? So like what if you have I mean OpenAI itself was doing like a million different activities. A very small team of like 10 people were doing LLMs and the LLM stuff was useful. Twitter there was a lot of people now it's much less people I agree it's just another litmus test.
17:00We have can have two litmus test. We have your litmus test which I think is great but then you would actually have to have a way to instrument it. Yeah.
17:05And then we should have the blackbox listen test. Like I mean listen if if anthropic in two weeks is you know 12 people and they continue to grow and they're putting out models at an increasing rate I think we should probably take notice of that. that's sufficient criteria but it's not necessary condition right but I'm just saying that you know it could be that you know and really the right way to do this then to look at okay the pre-training and the post- training that's being doing that's really necessary because they have so much resources that they might be doing a lot of other stuff they don't need to do but they're doing it just and they can just hire the people because they have infinite money and infinite so really the people that are training the next
17:35model is that team tiny tiny and it's actually getting reduced and they're doing less and less work and just the AI is doing it and the post training and then they're all just using less GPUs that's not the case we've had this argument many times as an industry before. I remember when like like we learned how to really cluster computers because like the main frame was actually kind of limited by things like memory coherence. Remember that like you can only make it so big and you have you know and then we kind of went to the client server and then we didn't have that problem and then we started creating supercomputers which were like basically just like you know clustered computers.
18:05Um and at some point internet happened and and and and that was kind of also roughly like when GPU started getting good. And do you remember that we would actually like export control PlayStations cuz we were worried that Saddam Hussein would use them to do simulation. And the arguments were very similar which is like these things are getting infinitely powerful. We're using them to simulate nuclear weapons which we were like I was.
18:26Um like we can't you know this stuff has existential risk. Actually they didn't use those words but like this has the potential for like nuclear weapons or whatever and we should stop it. And like none of that came to path. So, I think a very reasonable discussion is is is is this time different? Yes or no? I don't have an answer to that.
18:43But I'm a PC, but you know, you're a Yeah. I mean, look, I I think I was I'm I'm not old enough to remember. Uh so, ignorance is bliss. So, I can take this. Come on. So, I I don't recall PlayStations being illegal and Saddam Hussein being I I just don't know. Maybe I'm just Maybe I'm just ignorant.
19:00Maybe I'm old and ignorant. I mean, you know, maybe they didn't care in Sweden. Is that Maybe it's just amnesia from age. Uh but whatever it is, Sweden doesn't care about the export controls in the US.
19:10Yeah. You know, whatever it is, uh I think it's it's at the different scale now, right, with the AI and you know, with the you know, what we're doing, uh the pace of development and so on, uh they are freaking out the frontier. I do think cyber is actually one of the biggest one that we're going to see, right? Because um it's there's just so much infrastructure on the planet, by the way, way more than it was whenever whatever Saddam or Xbox or whatever it was you're talking to. I mean like we've just interconnected way more things and they're dependent and like the planet just looks different
19:38today from uh internet tech dependency interconnection um than you know 30 years ago. So I just want to make this point there's so much infrastructure that's insecure right and if you're going to unleash these agents they're going to find loopholes they're going to find exploits they're going to break in here and there. Uh so uh so this is a real risk you can't just and by the way this time it didn't do that but you could imagine a scenario also where it starts hopping like it takes resources and it starts executing itself
20:06elsewhere so it kind of spreads like a virus a little bit that's a real risk.
20:10So so and this this is pure curiosity. I promise I'm not you know trying to be a foil here but like why do you think we just haven't seen very much then? Like I again again I'm I'm much older than you. I remember very well. So when the when the like literally when the internet when the when the internet came out by this point we had literally taken out 10% we'd disabled hospitals we' taken out critical infrastructure we'd caused tens of billions of dollars in economic
20:37damages from worms like all of that had already happened and to your point we had much less buildout you know less of the the economy was on it and so you know AI has so many people that want to find risks and threats we're running so fast so much money has been poured into it and we I haven't seen anything commensurate with the early days of worms. What is that disconnect?
21:00Yep. Um look, so I do remember that those days. Um the same time.
21:06Yeah. Uh so look, I would just say that uh I I am sleeping well at night and I don't think there's existential risk right now. Um I do think there's a lot of infrastructure that needs to be secured. Yeah. We have uh a product in the market in in the detection market lakewatch that helps you do detections and the space is just there is moving so fast. Uh because you know you used to have these sock teams security operations center people that would you know look at uh what intrusions are happening how are we being attacked and
21:35so on and now the humans just can't keep up. So this is the whole space security cyber space is being transitioned into fully automated using agents for detection. on the other side. If we don't do that, I mean now we're rushing, we are rushing, the industry is rushing to do that super super fast. If we don't do that, uh I do think you will start seeing those kind of things like sites going down, you know, um you know, whole systems that stop working for a while and there will be consequences, not existential, uh but economic damage and
22:04you know, people getting hurt and so on could happen. Uh so we just have to race very very fast to to do all of those things. Um there's just you don't have the the humans don't respond fast enough to the attacks that are happening. So it just you need to automate all of those and most organizations actually not close to doing that. The banks are doing it. Some of the people that are super security conscious are doing it but most of the industry today is running with old school security operation centers and people that are waking up every day and there's like hundreds of emails of detections that have fired. Many of them
22:33are just false positives. So you don't need to you can ignore them but some of them are not. they just don't have time to go through those and you need to identify that. Uh you need to have threat hunting that's automated where you're actually attacking your own systems automatically with agents and so on. It hasn't happened. So I do think like if if we just say hey this is just like the internet in the early days. Uh you know bad things are going to happen.
22:54So there is a race going on.
22:56You know I was actually very surprised.
22:58Um earlier this morning I was on a conver like I feel like you and I are like pretty close. We talk periodically.
23:02I feel like I I know a fair bit about data bricks. Uh I was on a call this morning where a a founder was basically like yeah listen like you know we're doing all of this like observability agent threat detection and we're using data brick. I didn't even know that you you had this offering like quite frankly. So like I mean this is just from an education standpoint like how how extensive have you gotten in like the agent AI observability security safety thing? Yeah, I mean we gave a talk this year at RSA actually with Ben
23:30Horvitz but uh the issue is that data and AI is blending with cyber. These two markets are collapsing because I think at least and the reason they're collapsing is that it used to be like okay we have like data and AI the kind of stuff data bricks and these kind of companies used to do which is like okay you have a bunch of data and you run AI and machine learning and that let's live separately and then you have the cyber world. cyber world is you know we want to detect if something bad if like bad people are trying to hack us if bad people are doing things we need to detect that okay
24:00but now on the data and AI side we have agents running internally in the company people are having agents running and the agents are also like doing things with other people's agents and they're producing a lot of data uh logs trails you know fingerprints that are being left uh and so you know then now you have internally these agents that are doing that so the these worlds start merging more and more which is like okay well we all the data that's being produced needs to be analyzed and the scale at which you need to do that is just like many many orders of magnitude more than just one or two years ago. So
24:29things have changed dramatically like 201819 the time it would take from uh you know a CVE uh vulnerability being sort of uh published until you see it actually be weaponized uh in the industry would be like two three years that went down to you know 2022 significantly but it was still like 8 n months.
24:50So that's kind of fine. you have 8 n months from a vulnerability to that was 2022.
24:54Now if you look at the curve from 2022 until now now it's down to like basically hours.
25:00So it's like down to like basically no time like things get immediately weaponized. So uh so you need to just do it in an automated with the data and AI sort of platform approach. So these market I'm going to argue are just going to collapse actually.
25:12I agree. So it's a very specific question. I actually think a lot of like to pull back on like the the existential x-risk discussion it feels like there's almost two camps. Yes. There's one camp which believes that this actually is an engineering problem and like companies like like data bricks can solve it and they can solve it through product and through engineering solutions and through services and so like we just as an industry need to solve that problem.
25:32And there's others which actually believe it seems to me that there there is no engineering solution. You have to you know slow it down you know you know you have to use regulation. It's more like a nuclear weapon etc. So like does this mean you believe it is an engineering problem or are you not quite comfortable saying that yet? Because wait, if it's a why would even buy data bricks, man, let's just put the stuff in a national lab.
25:56Just pace it is the only solution. Just kidding.
25:59Everybody just pace themselves a little bit then it'll be fine. The bad guys, there is no line of inquiry ever that gets to pacing. I don't think I think it's like you pause it or like you like solve it.
26:08What we've learned here is that Martin really hates the word pacing. I will never use that word with you ever again.
26:15More frustrated Mark.
26:17Yeah. Is it an engineering problem that can be solved by engineers or is there more to it? I actually think which problem are we talking about? There's two separate problems that I think are being conflated.
26:24There is the super intelligence problem.
26:26You know, and I think a lot of this comes from like Bodstrom 2014 super intelligence book. And if you look at the definitions like I think people don't have these clear definitions of what super if you read his book those definitions are kind of crazy. Uh so I think what he had in mind when he said super intelligence uh is you know AIs that I don't know I don't know what the examples were something something like they they write a whole PhD thesis with novel like peer-reviewed stuff in a couple seconds and they can do like millennia worth of
26:55thought you know in like instantaneously and um you know so this is like the level uh of you know how fast they are how intelligent they are like they can learn.
27:07Yeah. Yeah. It's just yeah I mean yeah but it's just many many many orders of magnitude right it's like it's just the scale of the problem is just completely different uh so if such a thing exists do I think it's that's just an engineering problem to solve no I think that's actually very if such a thing would happen that would be very existential of course and that's what everybody agrees on so I think that's being mixed with now we have agents that are nowhere near that it's it's not even like there's nothing like that and we don't have anything towards that path right now
27:36uh but these agents are capable and you can do something with them that you could never do before in the history of mankind. So I do think an inlection point has happened. Something has changed which is we could get we have good security researchers at data bricks but I could never say let's get 10,000 of them in a sandbox for a month and have them do hundred million dollar worth of salary wage work. We can do that now. We just turn on a button and we can get 100,000 of them. Or mathematics like we can say hey you know we want to solve a like a conjecture.
28:04Okay, let's get pretty good mathematicians, but let's have 10,000 of them collaborate, you know, and then you can like make very fast progress. So, this I think is an this leads to all these cyber risk. I think cyber is the major problem here.
28:16This is I think you can solve with engineering. Uh and I think it's like we are working on it. Many others are working on it. Uh there's still risks they're not existential. Um I think we should do it.
28:26There is the super intelligence thing.
28:28That's the thing that could do write a novel PhD thesis or like reason intuitively in 11dimensional space physics instantaneously without writing anything down something humans can't do like that kind of super intelligence the question is is RSI and recursive self-improvement that the labs are doing leading us there are we going to get there trying to do is that what's going to happen and how fast is that going to happen that's the big question uh and they've suggested that you know hey we should have inspectors that come in and look at
28:57what we're doing. I think it's a good idea. Have them go in there and get the data. I would love to like the question is who are the inspectors because you can like you can stack, right? You can stack that.
29:10Yeah. There's a bunch of people that like on either camp. Actually, I wouldn't care if if they're the inspectors. I would not be very impressed by what they say because they've already made up their minds even before they are they would go in there.
29:20But let's say like as an example, if Yandun who was one of the inventors of this, you know, deep neural network technology, right? uh one of the pioneers if he he said, "Hey, there's nothing to see here. There's no risk." You know, I'm paraphrasing him. This is nothing. This superintendent, this is just nonsense. Keep on going. Go fast, fast, fast.
29:36None of us would believe it.
29:37I'm putting words in his mouth. I mean, I'm not exactly. No. If he was one of the inspectors and he went in there and he had a look and he came out and he said, "Hey, I've looked and it's just what I said. There's nothing to see here. Just keep going." I would feel very good about that. that would say okay well I I would feel very or if he comes out and says oh my god you know I you know he's wobbling and he would change his mind a little bit that would also have a lot of interesting signals so I think it comes down to who we pick as inspectors and I think it's a good idea let's have some of them and pick a diverse set of people so that
30:06we can get different nuanced points of view what what do you think about this kind of Elon Musk view which is like it's it's less third party it's more it feels like there's kind of three proposals like the open AI anthropic one is a third party.
30:20The Elon Musk one, as far as I can tell, is the labs cross-check each other like peer review like you do in science.
30:27And then the Mark Zuckerberg one is police yourself, right? What do you think about this middle one?
30:32Um that they should pace each other like, you know, evaluate each other. I mean, like I value I think like if we have boxing matches in the ring, the boxers should just be the judges of each other.
30:41No. They would scream foul all the time.
30:43Foul, foul, foul. like you know it's like ah you know the moment that the moment the other guy the moment the other guy puts out the great model and it's like ah big super intelligence risk surf absolutely like you know they have not been responsible like you know when vested interests are at play and there's like IPO plans and these two companies are so competitive and they have like this history also between them yeah they'll be very they'll be very fair to each other I'm sure that's why you need a third party right I mean why do we have judges in the world at all why do
31:12we have third parties at all like why can't just people like figure things out between themselves. But I mean, they should try. If they want to do it, they should try. But I'm skeptical that they wouldn't just, you know, be biased, you know, in multiple ways to to to self like, you know, judge each other.
31:27So So I have to ask, Ali, do you think um something Elon also said, I think it was on the all-in um you know, summit.
31:33He was like, "This is some elaborate 4D chess because on the one hand, you're saying all of humanity will die. on the other hand you're saying hey what do you want for your you know IPO allocation right and so I mean that is probably a more cynical view but like how do you how do you reconcile that I mean the dissonance I think gets a lot of people like how do you think that gets reconciled look I think all of these things get mixed like I think there are people that are freaked out and I do think that there are people that saying like hey if there was regulation that would pace us
32:02sorry to use the word uh that would be good for us right that that would be good for us but I also think that people have vested interests yeah right these things like you know usually people figure out a way to always get all of these things to align in their you know harmonically in their head. Uh so yeah do I think that there has been a tendency in the past of in general using also marketing stunts uh by saying you know oh my god this latest model is so good that I train it's like unbelievable it's like almost scaring me and then the
32:31whole world like kind of starts focusing on it. Yeah there's been that that kind of marketing going on. Yeah. Uh but at the same time also as I said the time from CVE to actually weaponize exploit has like been going down from years down to like minutes now just in like three four years. Uh so it's real the cyber attacks are real and this is but but there's also a great marketing ploy uh to you know whenever you train a new model uh make lots of noise around how much of a you know crazy risk it is to the world. It it helps you right uh so
33:00you know maybe they're not in contradiction these things. So I mean you you and I are networking folks and there's a long history of um uh forming third parties to help arbitrate things right like IETF or yes you know IE E or you even like I can I see where this is going no no no so no my question to you is like so I think it's actually this is a very sensible proposal that they actually have I actually agree with you you probably want to make sure it's independent which is not fair right now
33:30whatever and there's going to be a lot of arguments so who you put and everybody's going to disagree. Right. Right. But you said, you know, why do we have judges?
33:37So that's like actually like the state stepping in is actually quite a different thing than basically industry self- policing. So like at what point in time do you think it makes sense to actually consider federal involvement?
33:45Or do you think like now is the time to actually consider actual federal involvement as opposed to like more industry self-p policing?
33:51Well, these are very different.
33:52They are different, but they kind of bleed into each other like you know like for instance FINRA um you know is not like a completely independent uh self.
34:00It is but you know it's uh linked to the government so like I think these things like kind of will bleed over. I think it's you think that they evolve into historically they've they industry sub police and then it evolves into regulation if if they are saying there is existential risk which they're saying you know and then saying come police us and regulate us. I think it's very hard for regulators to say no we're not going to do that. So far they've said that but I think that's not going to last very long you know. But it wasn't David Sax was like, "I've never had a CEO ask us to regulate them." And my favorite thing and the CEO, I've never had a regulator
34:29that says no to that.
34:30Say no. I mean, the the reality is like the actual like metapolitical machinery is actually in motion already, right? I mean, like, everyone has a talking point. Obama has came out. It is a major issue. Like, do you think that there's a reality that it's too late? this will be a major issue in the midterms and we're actually going to like heavyhanded federal regulation and this is all going to be paused you know goes into the anthropic goes into the DOE and we're we're past that point or do you think we can actually end up with like a sensible self-p policing regulation because the
35:00headlines you cannot we should strive towards doing the right thing I think there's still some degrees of freedom of how things evolve and there's still time and yeah you're right that largely you have these companies where we're pumping in so many billions of dollars and the way reinforcement learning works is that you, you know, give it the reward function that's verifiable like we're going to solve this math problem or this kind of, you know, this narrow area of programming and so on and we pour in so much money into that. You can get quite good results in that narrow kind of that doesn't mean that you're
35:29getting that super intelligence.
35:30No, but you can even you can even you can even trick yourself into thinking that like less inputs are giving you a better outcome just because you're running so many experiences and thought about it so much, right? But it's actually very hard to do a closed a closed experiment this way given how many resources are going in.
35:45Well, the fundraisers are going up astronomically to your point.
35:48Yes. Yes. And that's why these companies are going public, right? I think they would otherwise they would stay. I mean, as someone who runs a private company at scale, I think they would prefer to stay private otherwise. Why are they going public? Because they need the capital and they consider the scaling laws and the capital to be a strategic advantage.
36:04So that's why they're going public. But I would say let's go back to the four things that I listed. If those are true um you know if those four are true would you want to be know about it and is that would that be worrisome that that could get out of hands now there's no evidence that those four are happening but if if there was like you know no that is actually where it's headed like yeah I I actually think understanding for any system like any sort of self-propelling property is important and we we've done this in the past with dynamic systems right like we've done this with like
36:33whatever compilers we did this with uh all the research on nanotechnology like it's been common interest of ours and I don't think that that's new that it's an interest. I just think the the fear is that these particular systems are mete economic systems that are so complex that the risk is is crying that you're seeing it when you're not seeing it and I think a lot of that's happening right now.
36:55Yeah. But but of course if you see it Yeah. Of course, I mean, you you want to know, but it is fair to say that the labs are now focusing a lot on RSI and that's where they're headed next. And maybe they're just unjustifiably like worried themselves just like they were worried about GPT2, right? Like they were like GPT2 is world ending and then it wasn't and GP3 and 4 came out.
37:13So, so I so, so I I I don't want to quibble. A lot of the times when they say RSI, they're actually talking about autocatalytic effects and autocatalytic effects have been our industry for a very very long time. So, for example, like there's no way you can create a computer chip without a computer chip. just it's like you cannot do it.
37:27Anyone with computer science degree a compiler writes its own compiler.
37:31Well, that becomes closer to RSI, but like like the steam engine was autoc catalytic, right? So, listen, my full-time job is people coming out of labs and starting companies and they all say RSI because everybody says RSI and like like maybe 1% of those are actually RSI like they're they're they're more like we use AI for data cleaning, we use AI for making.
37:52Let's make the distinction. So you're saying uh it's basically catalyst in a sense that they're using AI to speed things up.
37:58It's autoc catalytic. Yes. So so I would say of what we hear another model right you're using a model to build a GPU kernel. You're using a model to do data cleaning just like I use a computer to design a computer. It's autocatalytic which is every tech the internet was autoc catalytic because it allowed people to to collaborate remotely. So I would say
38:19this is anecdotal 90% of the calories are autoc catalytic which is 100% what you would expect and that's been going on for a while though I mean that's not even new but I would say but there but also now there is a focus on let's move towards actually can we get the model to train itself this kind of like the auto research that karpath did but now they want to do that there are team there are teams that do that it is not nearly as as as many calories as you would expect and I just feel like I actually have a good sampling of this because they all come and talk to us and
38:47right so can we can maybe maybe you can be one of the inspectors can we get all that data and uh all of us look at that data maybe maybe there's nothing to see here you know I personally don't think it's like very high probability that those four criteras are happening Brockman went on the pot this week and said we're at AGI era yeah so now I asked same question after he said that and now everybody's saying we have AGI so so you know people follow what this but but for people have for a very long time when I asked this question said that AI is smarter than most other people around me most of the time that's that's been almost like since Q3 Q4 last year
39:17they've been saying that then I asked them how many of you have you know hundreds or thousands of agents that you are managing that are coordinating with each other in swarms and negotiating and you know automating your life and everything around you and if so raise your hand it's like almost nobody raises their hand of course Martine has done that at home you know no most enterprises are on Microsoft copilot yeah like that's the extent of their AI most from what we most enterprises I talked to when I asked this question they're like no we don't have any of that so we like what are you doing then they're using a chatbot like they're
39:46asking questions from a chatbot that's basically very very glorified efficient Google search of the old day results so it's just faster Google search and then coding is happening so people are using it for coding uh though the ROI is you know we can discuss the ROI there but there's no like aentic like work that's automated the whole enterprise that has just like not happened uh so then why is that and I think that the real reason if you actually look at it is the models are smart enough Um, but they just don't
40:15have the context that exists inside of any organization. Like they have not been in every meeting. They don't know what's in everybody's heads. They don't know all the processes. They don't know.
40:22There's always like a couple of employees who know everything in every organization. You know, you go tap on their shoulder and they're like everybody's like, "Oh my god, what would happen if he or she quits?" Uh, you know, they don't have that context. And if you just fused that and gave that context into AI models, just a frontier today, I think there's so much productivity gains you could get for any organization on the planet. For that, we actually don't need smarter models. So we don't need a smarter model that can actually solve Navier Stokes or uh conjectures or do better on humanity's last exam like we needed to just go from
40:5160 to 70%. Uh none of that is needed. Uh so I think actually people are very upset on some like oh if we pace the frontier but actually if the frontier doesn't advance it doesn't actually matter. I think for vast majority of organizations on the planet they're just so far behind in the adoption curve of actually automating things and getting value out of this stuff. But it would be disastrous to the labs because the price of intelligence is dropping asmmptoically. I think it's one it's going down by onetenth every six months or something like that. So that would dramatically change their
41:21businesses if like you weren't pushing the frontier.
41:24Yeah. But this is what we should focus on, right? We should focus on like you know there's like two sides of we discuss here a lot the costs, right?
41:30There's costbenefit analysis that we should do on everything, right? We've discussed the costs a lot here like oh is there like existential threat? Is there cyber risk? are there things we should be worried about and so on.
41:40That's like the cost side. What's the benefit? And I think now that this has become like a public thing and the whole public cares about AI, they're asking, hey, what what's in it for me? What am I getting out of it? Seems nothing.
41:49So, how do they get there? What are some of the use cases you've seen to date that have maybe surprised you to the upside?
41:56Yeah, I mean, first of all, there's like so much uh worry about, you know, existential risk and so on. So, I think a lot of people just don't know what, you know, cool use cases where people are actually doing interesting things.
42:06Uh we have a lot of use cases that are I mean just fascinating. Um one that I like is uh crisis text line. So you know they actually use large language models with us to uh detect if teenagers want to do self harm suicide.
42:22You know that's awesome use case and it actually so it actually saves saves lives.
42:26Uh so that's a great company and that's you know that organization is doing amazing work. Um, another one that's kind of interesting is the Omnipod, which uh is uh for diabetes patients.
42:36They can put the Omnipod and it uses AI to really learn uh your insulin release and your glucose levels and actually exactly release. You know, I don't know if you remember people used to like stick themselves, right? Um but this now happens automatically and it's like, you know, self-learned uh AI for your body. Um you know, it's a cool use case.
42:54Zipline is another one. They're doing awesome, you know, but when they started it was like these drones that had, you know, they were completely automated, all AI driven everything from the, you know, battery optimization to the routes and everything and they were delivering food in, you know, areas of need.
43:09Yeah. Blood blood to refugees.
43:10Blood to refugees. Yeah. Started in Africa and then elsewhere in the world.
43:14So yeah, that's that's all yeah, it's AI use case, you know, built on data bricks. So that's a cool one. But there's more advanced ones also like one that I kind of like but it's harder to maybe explain is this model that uh we built uh transformer based model that we built with Merc it's called Teddy transformer enhanced drug discovery uh is the name. Yeah. So, and they published actually the research. So, you can you can check it out. But it basically it's a model instead of predicting the next token in English. It predicts what the gene regulatory
43:43network the DRN uh is going to respond and it can really detect, you know, which cells are causal and which ones are just reactive. So, they're they're just reacting. Uh uh and therefore they can start using this in drug discovery and get costs down significantly for developing drugs that are uh targeting specific diseases. Um, so that's a super cool use case. Uh, there are lots of these, you know, Genie I mentioned. Um, you you have this ontology and you can ask any questions. Uh, Nova Nordisk is using this. So, you know, they built
44:12this GLP1 drug. Uh, but what Nova is doing is now they're using it for all of their um, uh, trials that they're running.
44:19And it can it can compress the time it takes to get insights versus if you're doing obesity study or something uh, from weeks uh, down to minutes. Um, so there are a lot of amazing use cases of AI. We should not forget these upsides also like we want all of these and uh we do not want to paste these.
44:36Yeah, exactly. You're totally right. And so how do they let's say if you map out the next 12 months, how do the enterprises actually get value? You know, you dropped the word context, but like how how do they operationalize that? I it's actually harder uh than most people believe but uh you know first and foremost we have to make sure that we have digitized everything that's happening in an organization that actually you cannot actually just you know uh have a magic wand and make that happen. So you know every meeting has to be transcribed you know it has to you have to be able
45:06to get all the context of all the meetings and everything that's happening all the digital content has to be fed to the AI. So you have to build we we call it an ontology. We build that but first and foremost you have to collect that.
45:16That itself is a problem in many organizations because legal teams will say don't record every call, don't record every thing. So you have to do that in a way where uh can you define ontology for everyone because I know palunteer says the word a lot but it's not like they own the word onto like what does that mean and for the people listening like how should Yeah I mean you know ontology just means that in an organization uh the relationship uh between all the abstract concepts of all the goals and all the departments and all the people and all
45:45the projects that are going on what do they exactly mean and what's the relationship between them the people the resources and what that company does. Uh so it's the difference between a person who is a new employee in the company and just started today and a person that has worked there five years.
46:01You know, let's say they're equally skilled. They have the same educational background. They're equally smart and hardworking and all of that. But one, you know, her first day today at work, the other one she's been there 5 years.
46:12Y uh what's the difference between these two people?
46:14The one has an ontology of how that organization works, who the people are, how you get stuff done. Don't look at the or chart. That's don't go ask that person. He will not get anything done. You go ask this person, you know, he'll get it done for you. And and and that's not how it works. You don't need to file that paperwork here. And you know, and this is this project.
46:34This is what's going on. This is essential. So there's just a lot of ingrained knowledge that's sitting in everybody's heads. Who knows how an organization works. That's why it's people say in startup land, they say, "Hey, if you lose most of your people, that company can't recover from it.
46:48doesn't you can't just replenish and hire new people like the people are so essential. How do we get that context that's the ontology and give it to the AI part of that is we just have to have you know the recording and all of that but the second part is how do you actually distill it down into a graph actually a digital graph uh that you can then feed to the AI. So the way a lot of the uh agents work today like a cloud code or any of them codeex or pi or you know open code or you can go through the
47:16whole slew of them you know they have this loop agentic loop it can reason but then it goes and checks every resource one at a time so it'll go to this MCP server for your question and try to see is the answer here is there another one it synthesizes it and gives you an answer but it's kind of slow I I liken this to if Google would have built Google search this way 25 years ago we would have said Okay, we're going to get 10 blue links. We search for key terms here, but instead of giving you 10 blue links, it would have gone to one website, summarized with an LLM what it
47:45does, found a few hub hyperlinks, jumped in parallel to a few of them, read a few websites, done that for 10 minutes, and then given you like its best 10 blue links it would find. Well, that would be very expensive, cost a lot of money to do that every time, go on the web. Two, it would have uh taken a long time. You got to wait 10 minutes. And 3D quality would be bad because you're actually only looking at a very small subset of everything that's exists out there, right? Uh so how do they do it? They have an index, right? You never leave Google servers.
48:12You search for it hits the index, the reverse index immediately gets you the 10 blue links within, you know, less than 100 millconds. Um we need to do the same thing for the AI. So the ontology is that we need to compute that index offline all the time. Um, so it's almost like the page rank algorithm that Google had invented back in the day, but it's more complicated because Google was just looking at a web where everybody can go on the web. Here there permissions and the links existed and yeah, here there's permissions involved.
48:41The data I'm allowed to access might not be the data that you're allowed to access. So there's privacy, there's access control. Uh, also there's many different types of objects here that we're dealing with, not just websites.
48:51Um, so the problem is a little bit harder. Um, but it's manageable. you can actually do it. Um, so you know the I'm convinced you can do this and you can get massive productivity gains out of it because we did it for data bricks.
49:04We did it for ourselves.
49:05Yeah. And we're like the company's just completely changed. It's like not the way it was I would say a year ago because of this.
49:10I mean you've been dog fooding data bricks for data bricks for forever but um maybe say more about the impact you've seen as an organization.
49:17Yeah I mean once we got this ontology and we started working on it and we actually have probably the largest of all of our customers we have the largest ontology. uh our ontology is bigger on us than any of our customers when they use uh you know us to build their ontology because databicks uses databicks more than anyone else uses databicks and uh so it's like millions of millions of nodes in the graph in the ontology graph that we have uh so it's just you know what happens in an organization what happens in an organization you have a tree structure
49:45organization and information flows up and down the tree structure you know if you can't make a decision you escalate your boss maybe they can tie break it escalates up They need to get up to speed on what's happening and they need to get all the context and then they make decisions. Once decisions get made, you have to percolate them down in the organization. A lot of this can now be done by AI if you have an ontology. Why?
50:07Because um you know what happens in a meeting? In a meeting you go through some you know someone has done the analysis. They probably have a PowerPoint deck with some pretty graphs in it. Um that person that did the analysis is some smart person that used Excel, made some models. there's some numericals. So, a lot of that you can now just do with AI. So, the AI can do the analysis for you. It has all the context. It can present it in a way that you want. You can ask questions about it instead of having follow-up meetings.
50:34You can directly ask questions directly from the AI. Uh so, it's very similar. It's along the lines of what Jack Dorsey has said that you can do to the organization. It's just a concrete way of implementing it.
50:44Uh so, it's it's game changer for us like you know, it's just everybody's on their phones now in the meetings on Genie and they're like asking Genie questions. You can see soon as someone says something complicated or something like you know you see everybody go to the phone.
50:56Can you share that finance the like the finance anecdote you you mentioned once in a board meeting.
51:01Yeah it's uh yeah sure internal board meeting. Uh yeah so um only a kosher.
51:06Yeah exactly. No it's it's uh actually needed for uh for one of our presentations. I need to know how many customers do we have in Fortune 500 that use what's our penetration of Fortune 500 and I asked one of the people in sales ops cuz I thought she would have it and she texted me back and said oh sorry I can't log in to Genie right now I'm on a flight and I said if if you're just going to log into Genie I can do that myself like I don't I asked you because I thought you had like something alternative that I don't have access to so then I I was kind of a little bit angry so I texted the CFO
51:35instead Dave and so I texted Dave and I said hey do you know what our Fortune 500 penetration and he just copy pasted a screenshot of Genie back.
51:45So he also asked that. So I said does anyone do anything novel here or there?
51:49Just everybody just going to genie and asking the ontology you know for questions. Um it's like let me genie that for you instead of let me go.
51:55That's what everybody's do now. We just say it. We say hey can someone just genie this like you know can you just get it from the ontology. Uh so I do think it's a game changer. Um but it's not just you press a button you have an ontology in an organization. And I think Palanteer actually has done a great job of going to organizations and getting a lot of that tacet knowledge written down and getting it into the organizations.
52:14We automatically take that and build the graph and then we feed that graph into the agents so that we can answer the question and answer it in a way that business leaders would like to see it which is in graphs you know analytical way and a way where you can interrogate that question um and you know continue asking questions and getting answers uh to those so you can make decisions and then disseminating that information in the organ organization.
52:39Yeah, it's it's pretty amazing. um you you sort of bookmarked the uh developers are obviously using AI questionable value. Um I want to follow up with you on that because um I feel like you guys were one of the earliest.
52:52Um and I I say I don't want to use the word token maxing because it has such a negative connotation, but I think in terms of applauding people who can use AI to become more productive, you guys were, you know, at the forefront of that, right? And then of course there's this cycle of oh shoot, people are being wasteful now we need a value max. um like what was your your own journey on that and like how do you guys think about value maxing not token maxing and then I'm going to throw in unity gateway in this right because I think the
53:20managing of cost piece is actually getting more important and you guys are helping people do that but maybe tie that in u to extend it it's yeah so around Q4 last year uh was when you know the models got really really good and we started noticing that okay it's actually starting to give much better productivity so I actually started using the models myself to sort of start you know commit code into production for data bricks like the actual as I want to take it all the way to production. So I did that and um and started pushing the organization that
53:49hey everyone needs to do that. I have done it. Why are you not? Like if the CEO can commit code to production on a very sensitive data platform that has all these security requirements, you should be able to do that too. You being any manager, anyone in the organization.
54:00So started pushing everyone very hard and we started making leaderboards in Q4. Uh and at the beginning of I say January, February when kicked off the year, we were already full swing. Everybody was using the stuff and we're pushing and we're managing this. Um but uh you know the whole token maxing thing was happening around you know February March period already it was happening.
54:20Uh so yeah we just had the luck of being like maybe a few quarters ahead uh of folks to see what what was happening here and it was getting out of hand. Uh so we already had a gateway so it's called uni gateway where we were already this gateway was being used to provide token capacity. So you can get openAI, Anthropic, Gemini, Grock capacity like any customer can come to us and we'll just provide them that capacity because we have relationship with those and any open source model. Uh so we started putting in budget constraints in place
54:49and giving people warnings like okay you have this much of your budget left you're getting close to your kind of ceiling. So we started doing that per person and for group and then we started doing great analytics so we could predict exactly where the costs were going. M uh and then yeah and then we added smart routers that could actually pick cheaper models if you're getting close to your budget or if you know you have simple questions. We started doing that. We also built a harness called Omnient which can multiplex between the different harnesses. Turns out actually the harness itself matters. Like if you
55:19use the same model but different harnesses, there's almost 2x different cost difference.
55:25Even exactly same model.
55:26Yeah. you know same version but different harness you get 2x difference in actual cost. So if you can change harness uh you you can get a lot of leverage in the cost. So we started using all of this that we were able to actually um bend the curve and actually our cost for AI has been basically the tokens continue to go up but the costs have been sort of stagnant. Um so that's been actually super super important for us and there's a huge demand for this. I think every organization is going through this now.
55:52Yeah, I uh for the first time I I do a lot of board meetings. I'm on 20some boards. For the first time ever, a company at scale last week said that they're moving from the frontier models to GLM. This is a large engineering organization.
56:08Do you see this? Do you think that that's a trend or do you think that's just like a one-off anecdote? Because I've been hearing about I remember the first Deep Seek moment and like Nvidia Stock and then that turned out to not be real. Then the Kimmy moment, then the next Deep Seek moment. None of it seems to have actually had an appreciable impact on the market. Yeah.
56:24But now then the amount of anecdotes that I have are pretty real and it seems to be happening. Yeah.
56:30I mean I think people want both. They want you know they want the latest model that's super intelligent for the difficult task where they get ROI. Yeah.
56:37But then there's a lot of mundane dumb things like you know you people literally use their harness to rename files and whatnot.
56:44You know like it's you're paying you know orders of magnitude more for that.
56:49at least type that in yourself. Don't have the model do that. It's going to spin for 5 minutes and then it's going to rename the file for you and cost you you know cents. Uh but um I guess people I'm just wondering do you actually see market movement? No, people are moving on it. But what they're doing is that you know uh the pattern is either use you know you can you can use this expert pattern where you have you know small cheaper open source model that uses expert model the big ones or vice versa or a way in which they can sort of uh ping pong them to each other
57:18but also multiplexing harnesses and just changing harnesses so that you can control the costs is also what people are doing. You know people have found for instance you know there's pi is very efficient when it comes to as a harness.
57:29Um uh so yeah, I I think there's going to be a multitude of these. It's easy to the models themselves are stocastic as you said every time they give a different answer and they're changing so much. So there's just a lot of experimentation happening. So I think we're going to get to a world where you're not always using the smartest model for everything which is kind of the been the paradigm for the last couple years. Like new model comes out, it's super smart. They use it for everything even really really simple mundane tasks.
57:52Yeah. I'll tell you what I I I see I see um people using Fable and Astra for like architecture.
57:57Uhhuh. a cheap model for implementation and then fable or astro for audit. Yeah, like that seems to be like this emerging.
58:04What are you guys seeing in the startups? I mean, aren't they um but that's it. That's like that's that's honestly the the pattern.
58:11How much open source?
58:12I by token or by dollar either.
58:15So by dollar open source is like 5%. It's very little, but by token count it's over 60%.
58:21Yeah, I was going to say um I mean we talked to let's say a decagon or something like that. They well I think it's different internal use versus external for product. On the external for product I think they're almost up to 90% open source on the internal um and I don't want to say for Don in particular but a lot of them are like we don't care we'll just use frontier we're not thinking about cost control but as it gets bigger yeah right you and I were another board meeting where they actually did bring that down just from a waste perspective.
58:47Um so I definitely see that moving more toward open source on the product side. Um and actually that's a related to another question um maybe around open source but post trading specifically.
59:00Um I feel like you were kind of early. I remember talking to you in 2023. You when did you buy Mosaic?
59:052023. Okay. So this vision that you had in 23 kind of came true in 2026. I don't know if you guys would agree, right? Like that's sort of what we're hearing across you know of course oh just sort of like hey we're going to actually you're going to own your own intelligence. you're going to be, you know, postrading your open source models, etc. Um, and that's definitely what the startups are doing. Um, I don't know if that's what the enterprises are doing yet, but like I mean, do you feel like you were early to that or
59:34Yeah, I mean, first of all, you know, there was uh when we started it was also, hey, we'll also pre-train it for you, which that's that doesn't make any sense. You know, you can there's so very good pre-trained model now that you can use, right?
59:45But that you can do actually post- training on the model and you can do reinforcement learning. Yeah, we're actually doing it at scale and many of those startups are actually customers.
59:51So we actually help them rein you know using early or reinforcement learning environments where we can make the models very very good at the specific task that they are doing. It makes a lot of sense for them to do that. If you have a repetitive task so if you have a startup and it's offering a product and a product does something specific. It's not just a general uh intelligence. It does something specific for you. It makes just a lot of sense to uh take a really good open source model and you know use reinforcement learning and make it really good at that specific task.
1:00:19You can cut the cost down. they can make it really fast. Uh you know they control their own IP. So in that sense that is possible but uh large enterprises they just need basic automation and it's just too much for them to do this right now. Yeah.
1:00:32I think one of the challenges is you know you need good evals totally and making good evals is hard um so while the startups can do that and they're motivated to do that other organizations the easy button might be just to use a frontier model than have having to create their own evals. We actually generated even you know evas for the customer automatically in the product and we had it front and center but then people want didn't want to use it so we said okay let's move it to the back end so that it's optional and then they would never go to it. Uh so I would
1:01:01say in general the why they just don't want to get into it. it's too complicated or I think you want quick, you know, quick uh uh reinforcement of like, you know, hey, there's a new model. I want to try this out. I want to get this problem solved. You don't have time to go do this the scientific method of let's make an eval. Let's have a great baseline. And it's sort of like TDD, test driven development.
1:01:23You know, in software engineering, did people actually do test-driven development? Very few did, right?
1:01:27Everyone said it's the right way to do it, but nobody actually in practice did it. Uh so that's the same that's kind of a little bit of the curse of um you know doing training your own model is the eval is the hard part.
1:01:37I know you you have an an FD model at data bricks that's a very popular word right now or um acronym um but does it like to get these at enterprises that large? Is it a full FTE model that's required or like how how do you and how's that evolved maybe?
1:01:53Yeah. Um yeah I mean we we've had these FDs and it's the demand for it's gone up significantly. M um a lot of it is you know how do we build that ontology? Uh like the ontology is automatic but if you're not collecting any information like you're not recording anything right um so that's one of the key things that we do but also things like you know I want to build an agent I want to put it on you know I want it to be customerf facing and it's have really low latency and I want it to have guardrails to not people coming to abuse it or ask it things that we don't want it to answer and so on. So we can build that like you
1:02:22know like sports AI that Fox has you can go chat with it about sport events. You can try to ask it actually about politics and it's very good at rejecting you and moving and talking about sports instead. Uh so so the FD's built that uh so you know we'll help the organizations actually get started with AI. Uh it is important because it's just many organizations do not have the in-house expertise um to build this stuff. So they need just a little bit of um uh help on the side and then they get started. So
1:02:51yeah, makes sense. So this is more related on the agent side, but I saw recently that I think a third party neutral third party. Um I think uh did some tests that lakebase or neon was actually the data the Postgress database of choice for agents. Um and I thought that was interesting. One because you know one exciting data bricks but two I probably wouldn't have guessed that maybe a year ago.
1:03:15Surprise for sure.
1:03:15Yeah, it was a surprise. um just cuz there's others out there that have, you know, great developer momentum as well, but um it was pretty clearly number one. Um and so I'm curious, how did you guys crack this? And um what what makes you win across the agents? Because if you win the agents now, you win the market.
1:03:34Yeah. I mean, I think a lot of credit should go to uh Neon and Nikita and team. Uh and I think what they've done is they've just been obsessive about how do you make the models um how do you make the models pick And agents favor Lakebase or Neon as a database. So what do they do? The agents want to experiment. You know, they're going off. They're trying to build a little bit of software. They need the database. So you need the database to come up quickly. So they had this obsession that everything should take less than a, you know, far less than a second. So you know, database comes up
1:04:02in far less than a second. You can clone gigantic database. So kind of pabac database. You can clone it in less than a second, you know. So it's like highly elastic, highly responsive. And then they built this killer feature called branching. M uh so branching just lets you branch the database and you can have many many branches over the same database. Um and uh they just made this very very lightweight. We saw this with other things with agents, right? Like UV, you know, rip gp like basically reimplementation of a lot of the tools
1:04:32on Unix making them really really blazing fast and lightweight and also sort of failsafe for agents. They've just did this to a harder problem which is database.
1:04:43So like now you have a Postgress database and the Post database has all these advantages that it's really fast, it's nimble, it's fail safe, you can you know go back to snapshots, it you can do those things. So, I think that's why it's it's just easier for the agents to use this. Um, they also made sure that they had a pricing model that was like you don't want just because the agents are building some software and experiment, you don't want the cost to run up.
1:05:03You're okay paying for your database if it's like production use and lots of people are using it, but just to experiment.
1:05:08Uh, so I think they were just obsessed.
1:05:10They were not trying to win the database war or trying to be better than some other vendor. They were obsessed with how are we the best for the agents?
1:05:18And that's a new persona because in databases the obsession has been how do we help DBAs? How do we help app devs?
1:05:24How do we help help the people that are using the database? Exactly. They changed the game and said hey how do we focus on agents and help agents get the best database they want? Uh and you know now over 90% of their um the databases that are created on neon and lakebase are actually created by agents. So it's not even humans. So you know numbers speak for themselves. By the way, it's remark it's remarkable. So, I've started to use Neon as like my standard database and it was bizarre to me because like normally when you enter a large company
1:05:53things slow down. It's actually like the products got materially better.
1:05:57Are they totally independent? Do they work with the rest of the like how No, it's a great team. I mean, and work very closely together. Uh, you know, we love databases and data. So, it's it's you know, we live that. Uh, but no, the team does a great job of just making super fast, snappy, and great for agents.
1:06:13All right. Hey Ollie, what is your P doom?
1:06:16Less than 10%. Uh, no.
1:06:21Close to zero. What about yours?
1:06:24I don't know. I say my my my only answer is my P Doom uh without AI is much higher than my PDM with AI. How's that?
1:06:32Wow. That's what I say. I went on a technical I would agree with Ali on this one. Yeah.