0:00We're so traumatized by the dotcom bubble that we've priced in exuberance right from the outset. We will be spending far more on running models in 10 years time than we are today. Joining me in the hot seat today is one of the leading minds when it comes to Frontier Chips. Walter Goodwin, founder and CEO of Fractile. The business has just raised a $720 million series C collabed by Thrive Founders Fund, Light Speed and Radpoint, valuing the company at a $7
0:28billion price. It feels like we need thousands of times more compute than we have today. Frontier Intelligence is not yet commoditizing. I think we'll have a couple of multillion companies built in that space. One Neocloud that is competent is much like another Neocloud that is competent. Ready to go. Walter, I am so excited for this, dude.
0:59It's lovely to be able to do it in person and I'm so thrilled that you could join me in the studio. Man, it's very good to be with you, Harry.
1:06Now, I want to start for those that don't know 60 seconds. Like, what is Fractile? What do we need to know?
1:12Fractile. We're a frontier AI company of sorts. Um, but we make chips. We make uh these big rack scale systems um that are, you know, fully focused on advancing the frontier of AI capabilities. And so from a team composition point of view, this means that we have, you know, ML researchers down to uh process experts that fly out to Taiwan every few weeks down to people that actually look at the thermals and the mechanicals of putting together one
1:40of these rack scale systems.
1:43Totally get you. I was listening to you on No Prize. We were just talking about Sarah and you were saying unequivocally that you were a chip company and I was listening to this thinking great. But then I'm looking at it going everyone's moving full stack. No, we're seeing open AI have Jalapino. We're seeing Deepseek have their own. We're seeing Anthropic build their own. Obviously, Meta and Mark has his own. My question is, is the future won by vertical integration? And can you be a chip company alone these days?
2:10It's a very good question. I think there is there's a verticalization in that as all of these different components come together um and especially as you kind of look into the future and you work out, you know, what are these models going to look like? How do the workloads evolve? how does the silicon evolve?
2:26There is this absolute necessity to kind of have a vision on every part of that.
2:31Um, so Fractile is a chip company. Um, but in a sense, we're also a frontier AI company and I think what that means for us is, you know, and this will continue to be the case, you know, we monetize by selling chips. We monetize by selling racks of compute to our customers. But it's completely essential that we have a bunch of folks internally who are ML researchers who are looking at how model architectures might evolve who are maybe even running you know smaller scale experiments than the ones that you would
2:59see in the leading labs. And so I think there is a distinction here between the expertise that is needed internally to win and and the kind of nature of the business what the business model is. You know we we never intend to compete with our customers. we will never monetize internal models because that's how we've chosen to build this company. We've chosen to compete in the chip layer. Um but you can't sort of step back from
3:28challenging yourself to have expertise in all of these other places because you know in our case that would mean you you'd place the wrong bets. You have to synthesize everything that you're learning across the ecosystem and you have to bring that back then to be obsessive about the part of the space that that you're building. And if you do this right, you've ideally found yourself an enormous sort of piece of green space where you're incredibly helpful to your partners, to your customers, and to your suppliers
3:56upstream. Um, and you've kind of got to know parts of their business as well, but of course, you know, you never seek to to conflict with them. Would you argue that everyone's moving to vertical integration because everyone's fighting against commoditization of one layer? If you think about what you know, open eye did with Halipino in 9 months. Um it suggests some form of commoditization of the layer. Do you think everyone is desperately expanding to run against commoditization? I think there's
4:23definitely a fight to keep your margins expanded that do mean that let's say if you are serving frontier models you know it is important that you're not squeezed all the way down by very high margins on incumbent chip players and that's where it's very useful to have for example a first party play or another third party play that can be a counterpoint to that um and so I think in a sense there is this there is this kind of confluence of
4:51forces that do mean that you know if you're a frontier lab if you're a hyperscaler it's very important to kind of have your own retort to various parts of this um I do also think that there's this pretty crucial distinction today still between what is perhaps becoming commoditized and you know what we still see sort of at the frontier what is commoditized then help me I'm an ambassador okay I I I look for value in amazing entrepreneurs and businesses
5:20what is commoditized that most people think isn't and what isn't that maybe people think is.
5:25There's this sort of sometimes concern, sometimes excitement about the idea of intelligence saturation. You know that as models get smarter and smarter, there will become a kind of waterline that they exceed where we don't really need greater intelligence for a whole set of tasks. Um and I think it is true that there are certain tasks for which we see this kind of intelligence saturation and this is where you then see you know an open model is good enough uh and these models become commoditized at that
5:55layer. Um I think where that becomes more nuanced though and where actually models even at that level sort of still fight against commoditization is that there is always a kind of economic pressure to for example reduce the cost of intelligence. uh and perhaps counterintuitively you know this is actually why I think the frontier labs and the frontier efforts remain so vital is of course frontier intelligence is not yet commoditizing.
6:25We see you know large margins there. We see big deltas against you know against sort of everybody else. Um and also these sort of learnings from the frontier they kind of trickle down also into building better you know better smaller models as well. And so actually you if you look at that sort of fight across the entire gamut of Frontier LLM capability as well as kind of behind the frontier LLM capability there's still this sort of ferocious battle. Um and I
6:55think you know I would distinguish or I would characterize that as as sort of not commoditized in so far as if you are a market leader in something uh you can still extract high margins. So a good example of this actually is DeepSeek released a few days ago their entire like inference stack, their entire software stack and it had really remarkable, you know, people very quickly characterized it and realized that Deepseek had done so well in building an inference stack to run their
7:23models on GPUs that they were getting, you know, much higher throughputs, much better interactivity, better speeds than than everybody else who had built a Deepseek inference server. And so you know something that you might have thought would have been by this point commoditized which is the serving of an open model. You know in their case by going full stack uh they built a model which was more efficient and they built an infant server for that model that was more efficient and they'd carved themselves out the right to have you know really healthy margins on that. So
7:52I think even in the sort of open source space we don't yet see this kind of complete race to the bottom at least for those folks that sort of again try to own a lot of vertical understanding of the space. I think what is commoditizing and I don't think anybody has ever argued it won't be is kind of you know the whole space of of cloud compute. You know one neocloud that is competent is much like another neocloud that is competent. Uh the reason it's so exciting today is we don't yet know
8:22which ones are competent. So I think there's still a lot of space there for there to be huge winners and and maybe sort of huge losers as well.
8:29But I think that's why you see all the neocclass desperately trying to vertically integrate too just because I think they're very aware of the commoditization of their own layer. You know we had the founder of Nebius on the show who was talking about the movement into model rooting and inference for them as a core business and a core strategic just because I think they're aware of that. Yeah, I think that's true. And you definitely see, you know, every NeoCloud is trying to build an inference server. Uh, a lot of these folks who built an advantage in sort of inference serving, you know, which is a software advantage ultimately are also
8:58thinking about how they can have more control of their destiny on the compute side. Um, obviously I'm biased, but I think this is where the kind of neochip story and the, you know, the differentiated silicon can actually become a really core differentiator for those players as well. Can you unpack that? Why is that?
9:16So, we're still in the very early innings of the kind of roll out of silicon that is not the mainstream conventional, you know, GPUs and TPUs and other XPUs. Um, and it is still much bolder, especially if you look at the kind of way that these buildouts get financed to lean into, you know, a new chip platform, um, a Fractile, a Cerabus, and so on. And we need these new sorry just to bring everyone along with us. We need these new chip
9:45platforms because workloads are becoming more and more complex and intense and the specialization of these chips will allow for the execution of these workloads to be much more efficient. I think these new platforms you know obviously we're one of these you know have a belief that in certain certain workloads you have this kind of more than 10x advantage whether it's on cost or in our case it's cost and speed. Um and so you know in a sense they are contrarian bets still you know we still don't see many gigawatts of new silicon
10:15built out um but they do kind of have this potential to fundamentally alter the economics and so if you imagine you are a neocloud and you're the first neocloud to work out how to break out of you know a particular bond with a particular GPU vendor uh potentially you get to reap this enormous benefit and we've seen this with you know fluid stack is a great example right they were uh in some ways the first Neocloud to kind of uh decide that they were going to step uh bodily away from Nvidia and
10:44they deployed a bunch of Google TPUs and they've had an extraordinary year um as they sort of cross that Rubicon and earn the right to start deploying you know other chips um and so I think there are opportunities to uh you know as you kind of go more vertical also take some of these sort of slightly uh you higher risk bets I guess and and and and reap huge benefits from that.
11:11So if we think about that what does that do to the NVIDIA business? If we think about kind of Nvidia as a horizontal provider and you as a kind of specialized provider is the future of chips specialized according to workloads and how do we think about like that market composition horizontal versus specialist? Yeah, I think the way that I think about this, you know, Nvidia is an extraordinary business and they build by many metrics in a sense the best AI chips in the world, right? And the best
11:38systems and they have extraordinary work on data center scale inference for example. So in the infant space, I think they've done some some brilliant work as well. What I think is different about this moment in time and and and sort of caught to fractile's identity is there is this idea of kind of an evolving frontier and you know the frontier necessitates slightly different sort of bets and a slightly different way of
12:05thinking because you are necessarily setting up a business that is able to kind of look around corners, is able to make bold predictions and is then able to steer the entire tech product for example towards those bold predictions and I think if you look at Nvidia's business what they excel at is delivering highquality products at extraordinary volumes that are in a
12:33sense unopinionated right these are powerful generalpurpose processes that are uh very good at various parts of for example you know the AI compute world if you look at where fractals bet is on inference for example which is building chips that are orders of magnitude faster than these GPUs. You know, that's a particular focus. And for us architecturally, it's meant this focus on finding paths to huge memory bandwidth, for example. Uh, and it's potentially taken us away in some ways
13:02from some of that complete generality.
13:05Um, I think what's interesting though today and the reason of course that we're so excited about about this space is the frontier is kind of also an overwhelming portion of the total market. You know, if you look at what's happened just over the last few years, we have this enormous buildout of specifically inference compute to serve LLMs at the frontier. Um and so you know what is in some sense specialist is also
13:34by no means a niche. Um and I think that's this sort of real transformation that if you look back you know 10 years even 5 years uh it would have been slightly crazy to build a kind of specialist chip right we didn't have any single workload that made up sort of more than you know let's say 10% of the total compute burden suddenly we do and I think that demands slightly different company structures that can have that kind of predictive power built in that
14:02try to also orchestrate the entire org around what it means to to chase after that.
14:08There's so many elements there that I I want to unpack. You mentioned memory in HBM there. Why is the memory wall such a prominent problem still today? Can you help me understand that?
14:19Definitely. Yeah. It's a it's one of these sort of interesting facts about kind of chip physics, right? If you look at um the great days start with that line. Uh so if you look at if you look across the last like 20 years in in the silicon world um there's sort of two things that really make up the kind of top level specs the kind of top trumps of your chip right and like one of them certainly for AI chips is the flop count how many floating point operations can
14:47you do per second this is like a multiply or an ad um and so that's your kind of mathematical power on that chip and then the other piece is is kind of the properties of the memory on that chip the capacity and in particular the bandwidth to that memory. Uh, and that's how quickly you can, you know, shuttle data back and forth from you're looking at a GPU, you actually have that silicon in the middle of it that is running the maths. It's got that high flop count.
15:12And then there's other pieces of silicon around it, the HPM, uh, that are kind of coming in on this on this sort of highway, bringing data in. And we've been extraordinarily good with Mo's law at scaling flop count. We've scaled it about a million fold in the last 25 years on these chips. Um bandwidth by contrast is kind of much more intrinsically physics limited. We've scaled it about 40fold in the same time.
15:36So you know it's lagged behind the flop scaling about 25,000 times. And so we've naturally found ourselves more and more kind of bandwidth limited in a lot of the operations that we do. And for something like LLM inference, this has become a kind of critical bottleneck because of this interplay between the generation of text, which I think we all sort of have this uh sense that it's this sort of auto reggressive, you know, sequential thing. One word after another
16:06is generated out of these models. Every time I go and generate one of those words, I'm loading from that memory over that limited bandwidth all of the weights of my model. And today also all of the context that I'm operating on as well. So all of the path states of things that I've seen and it's that that is then my bottleneck on generation speed. You know the reason that if you take a very large frontier model it might generate at 80 tokens per second 80 words per second is going to be not
16:34because of the limited maths capability on the chip but because you don't have nearly enough bandwidth to that memory.
16:41I cannot load the weights of the model from memory into that processor more than let's say 80 times per second. Um, and the reason I started with, you know, the the sort of sexy line about physics here is it is if you zoom in at the interface, there's a bunch of essentially copper wires that are going from this DAM memory into the processor.
17:02And we've been pretty good, the world's been pretty good at making them smaller and smaller and smaller, but there's only sort of so much you can scale that. And so you have let's say a few thousand wires and is that why sorry you have SRAMM am I right in saying that and moving memory closer to it.
17:18So SRAMM yeah so there's this sort of um you know DRAM is like off in a different building basically and you're you're carrying these boxes down the stairs and then up the stairs again into the into the processor building where the maths is done and you have to carry them back again unfortunately because there's not enough space in that building let's say to to store all the boxes. So actually maybe it's a warehouse and then you've got the office where you're doing you're doing the work. Um SRAMM is like having a bunch of boxes that are permanently in in in in the in the processor, right? So
17:47that's a small amount of memory that you cram onto the same silicon um where you're doing your processing. Um but that's the that's the snag as well. The reason that over the last 30 years we've externalized memory into this this thing called DRAM and put it on different silicon. These are built in completely different foundaries. They've been specialized for very very different things. So DRAM is completely focused on density. The DRAM bit cell as it's
18:15called is an incredibly simple structure and you can cram uh you know many billions of these into a very small area of silicon. SRAM by contrast is built conveniently on the same exact processed nodes. You know it's built on the silicon that you're building your processor by TSMC for example. um but it isn't optimized for density. So you might have you know less than 10 times as much kind of SRAMM capacity per unit area as you can get sort of gigabytes of
18:44DRAM on that same area and by extension per dollar as well because your silicon wafer costs you a certain amount.
18:50When you look at inference and the future of inference which is then best suited is it DRAM or SRAMM because latency is important.
18:58That's right. So latency and and bandwidth as well like how much can I just cram through this? So this is one of the things that we've been working on over the past couple of years at Fractile is I guess the rejection of this trade-off. Um so SRAMM as a memory can know is the only one today that can be built on the same processor. It's the only one that can be incredibly close and have incredibly high bandwidth. DRAM much cheaper, much denser, much more capacity, but fundamentally limited in
19:26the way that we access it. But if you look at that that sort of physics, the fact that it's coming in over wires, it's because we've kind of got it in the wrong place, right? We've got the DRAM sort of sitting in plane with uh with the chip and we've got this like limited, it's called a beach front, right? It's like literally the limited kind of real estate on the edge of the chip. Um what we've been working on at Fractile over the past couple of years is uh essentially stacking that memory
19:54on top of the logic. So looking for ways to rather than have this kind of limited beachfront, you know, everybody gets to have a a seaside property. Basically, it's like the entire memory layer sits on top of that logic die and suddenly you're how is that possible?
20:11So 3D in in the chip world, it is a dimension that people have been scaling for a long time. Uh but it's got all of these complexities. So actually if you look at an HBM, a high bandwidth memory, it is a stack of DRAM chips. Um but it's a stack that has certain properties that you know don't make it amanable to just being taken and and plunked on top of uh your logic die and so we don't do that today. But there are these process technologies and this is something that
20:39sort of you know Fractile's team has a long hintterland in around uh 3D stacking. So, you're building condos on the beach instead of beach houses to allow for more capacity.
20:53I guess we're building one of those kind of uh you know, new built archipelos where everybody has a bit of beach front, right? It's uh it's the uh you know, strips of land. Everybody's right next to a piece of coast. When you put this on top, suddenly you have this whole two-dimensional area where you can have essentially those copper wires are now not going laterally. you have millions of them coming straight down.
21:18When you look at differentiation between other providers, is that your differentiating factor? And how do you analyze the specialized providers that also compete with you in this realm like an etch like like what is the differentiation?
21:31Yeah, I think there's a few ways to to look at this. You know, one is one is sort of philosophical and then the other is technology today. So I think philosophically um I do think that fractile is you know we have a mission right and the mission is to advance frontier AI capability through building better hardware um today that's taken us over the past you know past couple of years to this particular solution and I think this uh extraordinary work that the team has
21:59done in unlocking a future where you can have the bandwidth of SRAMM but with the capacity and the scalability of DRAM It is a worldchanging kind of innovation. Um, and it's going to have longevity as well. You know, we're we're very enthused about taking this forwards over multiple generations and kind of compounding a a technology advantage there. Um, but at the same time, you know, I think with a with a mission, if the workload changes, you also want to
22:27be able to to change your response. And so, you know, I think one thing which we're cautious not to become is the kind of, you know, the company that cracked 3D RAM. Um, we want to be challenged to continue to innovate on, you know, whatever comes next. If you look at mechanistically what it does mean for us though over the next few years, there is this sort of challenge today. If you look at the chip space where on the one hand you have all of these HBM based
22:53chips um and at some level it's very hard for them to diversify you know they are whether it's you know the GPUs TPUs the Japano chip from openai um etched matx you know they have uh often really brilliant kind of architectures on the chip which allow them to do things slightly more efficiently than one another and so on but they are fundamentally kind of uh boxed in by the like top trump specs, right? You can get only so many flops and you can get only so much bandwidth.
23:23The uh HBM that sits off to the side of them is exactly the same piece of silicon. It's the same module from the same vendors for each and every one of those chips. Um and then on the other side you've got these SRAMM based chips uh that have kind of high memory bandwidth but also have this huge capacity constraint right and so if you look at you know if you look at Grock or you look at Cabbrus what they can do today which is extraordinary is allow you to run powerful models much much
23:52faster one of the things that I think certainly I've always been excited about with Fractile it was sort of what I set the company up to do four years ago is build chips that are better at LLM inference, but in particular build chips that allow us to run that inference much much faster because we always believed that we would get to this space where we had what we actually do have today, what what's called test time compute scaling, right? Reasoning models, longunning
24:20agents that run for longer and longer to get better and better and better at what they're doing, you know, a better answer for you. Um, and this is the kind of profile of all of the frontier workload that's sort of changing the world today.
24:35You know, if you look at what happens when you turn a Frontier LLM to the problem of solving a Millennium Prize uh problem, you know, it will go off and generate many millions of tokens sequentially and you scale up this swarm of them as well. So there's a sort of parallel element of this, but the many many millions of sequential token generation means you're waiting days and days and days to get one of these responses. Um and it's a it's a space
25:02which somewhat tantalizingly hasn't been cracked at all by any of the silicon any of the chips that we have today because there's this sort of tantalizing mismatch I guess between the fast inference chips the SRAM based chips um that have the ability to you know access memory quickly generate tokens quickly and the needs of this new workload the kind of longrunning agent which needs a huge amount of capacity because you have these really long contexts that are emerging and so the
25:32world urgently needs this kind of uh up and to the right thing where you have simultaneously really high bandwidth and high speed but also high capacity so the ability to scale to longer contexts and larger models.
25:45Is this one of those where it always takes longer than you think or is this one where you see dramatic inflection points in the acceleration of technology? I think what's exciting about this one is and this is true I think often of of sort of you know technological breakthroughs you are always standing on the shoulders of giants um in our case you know some of the technology that we're leveraging to do this 3D integration for instance has actually been around for quite a long
26:14time it's just not been turned to this exact problem to the the memory stacking challenge and so it's a space where if you're willing to do the kind of both the architectural effort and redesign your chips to make use of this. And then you're also willing to do the kind of uh manufacturing operations leg work and you're able to persuade your partners that you're sort of not crazy along the way. There is actually the possibility here for these kind of overnight overnight transformations. Can I be
26:43blunt? I was talking to a founder of a a brilliant chip company last night and he said there's a huge amount of hot air and and BS in the industry and there's a lot of marketing fluff but there's not much actually in production when you look at your etched or your fractiles.
27:00Are we actually in production? Like how do we actually think about on the ground being in production? And is he right with that statement?
27:10There's two parts of this, you know.
27:11Yes, Fractile has silicon that's gone through the foundaries has come back to us. Um, yes, we're running further trial runs and we'll be doing sort of full product tape outs uh in a few months as well which is quite exciting. Um I think the scale of production is something which is very much worth challenging right there is a uh real challenge especially for kind of new chip entrance at the frontier um in accessing for
27:41instance TSMC capacity HBM capacity um and I think in particular you know for some of these efforts there is a race for exactly the same resource you know a lot of these chips today are built with the a backend partner for example you know whether it's the TPU or Jalapeno MTIA from Meta uh Matx or Etched um you know all of those are rumored to be using Broadcom for instance um and uh
28:09same or very similar TSMC process nodes same HBM technology and so you can see why there might be like a real kind of gold rush for that particular finite resource and I think that means that it can be challenging to get to really meaningful scales in a short space of time. And one of the things that we've sort of, I guess, always been quite focused on at Fractile is because we've had this sort of almost unfair reach into the stack, right? The the the point of a back-end partner is they do the
28:38kind of last mile, the physical design.
28:40They interact with the foundaries, with the with the manufacturing side of the business. Fractile's always had that in-house and we have this extraordinary team that has kind of always worked that way, you know, even before Fractile for for decades in how they've delivered their chips. Um and it's become a really core strategic advantage for us because it means we've both always had direct communion with our manufacturing partners. Um we've had to do that leg work from the outset.
29:06And just so I understand the alternative is you go through Broadcom who then have the manufacturing partner relationship.
29:12That's right. Got you. So Broadcom is the intermediary layer that most people work with that you don't and that we don't. And so I think there's there's two things.
29:20What does that enable you to do then? So one thing is build completely different kind of building blocks for these chips right I think if you look at a Broadcom uh you know I was at a conference this summer where Charlie Kawas who's who's uh you know one of the leaders there held up a kind of Lego model of the nextg platform and he said this is going to be in production with six major customers in 2028 which is a sort of strange thing to say because what he means is something that looks identical
29:49to this is going to be made with six of the major players and given different names. It'll be the TPU, MTIA, Jalapeno, you know, three um Dorito 2.
30:00Exactly. Um and uh and so it sort of reveals something about the structure of this market, right? Where each of those chips will have a different name. They will also be different. There are incredible chip teams inside all of those companies that come up with uh insanely clever kind of architectural optimizations, but in the end, it's going to be exactly that same platform.
30:20It's going to be the same stack of HBMs, the same number of chips. A lot of the sort of fundamental building blocks are dictated by the backend partner because it's a platform technology play. You know, it's a kind of you get to change the color of the paint basically. Um, and so for us, what it's allowed us to do is like maneuver and build these new building blocks. So the memory stacking work for instance is incredibly complex and painful and requires convening
30:49multiple you know much larger supply chain partners to get them to do something that is kind of highly custom.
30:57But once you crack through that as we have you know you'd then have an asset that nobody else has. And this was a slightly long-winded way of getting all the way to your question about like volume production. That also more likely than not has meant that you've gone and found a way to use resources that others haven't. So in our case, a different process node, maybe a slightly different DRM node or DM generation as well that might be going into Frontier HM. Um and
31:26so you sort of find you know the opportunity to build gigawatts of capacity sort of down the back of the sofa with things that other people and this sort of you know set of frontier platforms have already moved away from.
31:40Can I ask you when we think about the kind of production line so to speak there a lot of people told me that I need to ask about tape out what what is tape out what do we need to know and how how do you think about it? So tape out is the moment where uh the final file from your chip design process which is actually basically a it's like an enormous image we call it a GDS2 file and the image dictates you know where are all of the copper traces where are the transistors where's the logic that
32:08gets sent to to TSMC or to your to your logic foundry partner and they commence the process of building the mask sets the kind of stencils that will ultimately produce the chip and then you start doing your your first kind of trial runs, your first production runs as well potentially. Um, so it is a moment in time that suddenly if you're a chip designer is a bit of a kind of seinal handoff, right? Because at the end of this long process of architecture, front-end design,
32:36verification, physical design, layout, you send the file and if there's a bug in it at this point, you know, it's too late. The mask is getting printed. Uh, and so that's why it's kind of a uh, you know, it's a major date on the calendar sort of thing. It's also though just sort of today step step one in in quite a long process, right? Because that silicon then goes through the fab and you wait months for that to come back and you have to be poised and ready then to receive that and run, you know, your
33:05bringup process. You this package silicon returns to you in the lab. You then sprint to get, you know, a workload running on a single chip on a tray eventually on a rack as well. Um, and so that's sort of part two. I guess is that um you know at Fractile uh we track progress you know there's tape out dates in the middle of it but the thing that we're continually shooting for uh which we call task zero uh is you know the day
33:34that a particular reference workload runs on an entire rack of fractile chips.
33:40What does no one see about this process that is most prominent to you? If we think about tape out being like a seinal moment handoff, where is the god no one quite understands the difficulty the momentousness of the post tape out date?
33:56I think well I think that's it actually is it's what comes next because there is an awful lot that uh you can fail to prepare yourself for if you're ultra fixated on that one thing the kind of handoff of of the first chip to the to the fab. Um, and so I think, you know, for us it's continually tracking, you know, what is what is it that we're actually trying to build here? We're trying to build systems that yield well.
34:24So the underlying economics are great because, you know, we've we've designed silicon that has the right levels of repair and redundancy. It's, you know, highly manufacturable. It makes a lot of sense as a product and then fits into a system that again yields well, is reliable. Um, and so the sort of building of a product that is manufacturable in volume where you can then find a path to ramping as quickly as possible. You know, that's that's the
34:53goal in some ways. Um, and I think tapeout is a um, uh, I almost feel like maybe over the last few years there's been this sort of uh, accidental uh, shared obsession between sort of chip startup founders and v the venture capitalists that are backing them on this phrase because it's such a tangible moment. It's such a tangible date. Um, but a lot of the work that we do at Fractile, you know, all of the work we do on building software, all of the work that we do on systems and a lot of the
35:23silicon team as well is focused on those parts of the chip that in a sense are kind of like intangible, ineffable, but are about what it looks like to be able to get this stuff up and running and reliable and to volume as quickly as possible. When we were downstairs, you said that actually silicon maybe maybe no longer the bottleneck in the same way that it was before and maybe data centers were or data center capacity was. Can you talk to me about that
35:51relationship and the the transition of shortage versus not?
35:55Yeah. Yeah. It's been I mean this is why the last four years have been so exhilarating because it feels like every quarter uh the thing that is most constraining let's say our ability to run more frontier models at scale in the data center has shifted. Um there's these underlying properties that we urgently need like the ability to run inference much much faster, the ability to run at longer context with better economics. Um but then we've had this sort of you know if you think back 3 years ago there was no question of other
36:24chips it was an Nvidia GPU shortage. Uh and then I think we found ourselves in a underlying silicon and supply chain shortage. You know acute memory shortages today for HBM and so on which stretch years into the future. uh logic silicon shortages with with TSMC as well. Um and I think you know if you look at the space today land and power is definitely another limitation. You know we've had all of the kind of lowhanging fruit for
36:52industrial sites that can be retrofitted to be data centers. Um places where you know in the US you can get permitting to bring gas turbines close to spin these things up in you know 90 days. uh these kind of often one-time tricks have kind of all been exhausted. And so there's sort of we're now exploring the kind of longer tale of you know harder harder ways to find land and power to install these chips. And so on the you know one
37:20hand actually it is data center capacity that is our kind of uh fundamental bottleneck on how much compute we can build out. And so this is the kind of uh you know if you want to be completely agnostic and invest in a you know your long AI you know by buy by by energy basically because that is obviously the kind of most fundamental um but at the same time it's precisely that kind of constraining limit that is why it is so
37:48vital that we have the best possible chips for any given job. If you look at um uh Frontier Labs and Hyperscalers today that have like a fast mode on their models, uh you know, if you're running that on an HBM based chip, um you're doing so at this kind of extraordinarily inefficient point on a kind of trade-off curve. on these chips, you can either be sort of slow and efficient in generating tokens or uh
38:18fast but but deeply inefficient sort of far fewer tokens per second per megawatt of power for instance. Um, so what happens if I want to take a multi- trillion parameter model and run it at thousands of tokens per second? Uh, but I have a 500 megawatt data center and I don't have enough aggregate throughput.
38:37You know, I can't serve my user base on those chips. There's a real pressure to now install a set of chips that do retain let's say really good aggregate throughputs even as you squeeze them to to much much higher speeds. You know, Fractile would be an example of that.
38:52And so I think the uh although it's land and power that may be the kind of most acute bottleneck even ahead of kind of supply chain constraints, it's actually exactly that that kind of incentivizes uh chips that are much much more fit for purpose.
39:09How much of an energy deficit do we have? I know it's an incredibly macro and bold question, but when you look at energy requirements versus energy supply, what do you think the deficit looks like? I think if you look at backlogs in the US, in the UK, it's this extraordinary um you know the cues for grid connection will be like an order of magnitude more than there is energy in the grid. Um so viewed in that way you know and the extent to which all of those are actually legitimate projects that would
39:38get off the ground is obviously something that we can we can question. Um but it does look like there is this this staggering level of demand. Um, and I think energy is one of these real fundamentals.
39:50Going back to what we said earlier about DeepSeek and their incredible innovation on their inference stack, when you look at China's policy and regulatory environment towards energy, I mean, it's green light anything. It's green light anything, including moving millions of people out of their homes because they need the sites for energy provisioning.
40:09And in the US, you have nimism. In the UK, you have, you know, um, bats and butterflies preventing sites of scientific or natural beauty. Um, are we not just inherently disadvantaged in a world of energy requirements versus China?
40:24I think this is one of the places where China has, you know, has somewhat stolen the march on the west, right? uh and in particular, you know, these mega projects, you know, huge hydro projects, for example, they're ideal for AI compute buildouts because they have this particular kind of signature where, you know, it might have once been kind of inconvenient to have 15 gawatt of power all arriving at a single site that then needs to be distributed.
40:51You know, an AI data center doesn't really care where it's located. Um, and so why not go put 15 GW of AI compute right next to this, you know, hydro plant in the middle of nowhere. Um, and I think that there is, you know, in the US and in particular in the UK and in Europe, a lot that we can do to look a little more like that. And I think that the good news here and this is something which, you know, I think the government in particular, you know, here and and European government should be doing more
41:20to to play up is it's actually exactly that fact. It's that a lot of the power sources that could be brought to bear on this are um you know renewables can be a huge part of this story, right? You can locate a data center, you know, in in uh in Scotland next to an enormous wind farm where previously maybe there just would not have been a story as to why you wanted to have energy generation there. Do you know what's really interesting for me is we we often we do
41:48a lot of Instagram now in short form and the minute we talk about data centers and them not happening we did it the other day with Microsoft pulling out of the NScale project um everyone goes good riddance good ridds you tech bros good riddance and it was astonishing to see the universal backlash from bluntly moronic civilians can I ask you you you saw inference as a market before so many other people wish I'd listened years ago. The thing
42:17that I know now that I wish I'd known 12 months ago is just the infinite demand for inference.
42:23What do you know now about the market for inference that you think isn't that obvious or you hear other people say?
42:29I think the thing that remains uh in some ways it's a it's a even more pronounced version of of sort of my original bet. So you know I think four years ago the insight was sort of twofold. One is as you start to get these kind of foundation models that have been trained on everything and then generalized to anything you might turn them to. Of course you have this kind of hard logic where training becomes a fixed cost and the deployment you know inference is just the use of these
42:59models to do useful things. Um it's the only part that matters. If you were spending more on training in the long run like why are we even building these things? you know, you really could kind of pack up and go home. Uh I always describe it to my mother like kind of the Olympics in your same boat. You he trains in the gym, great, and then decides, I'm not going to actually do the race.
43:20What's the point, dude? This is why we're here.
43:23Well, it's like imagine you spent far more money building a factory than you ever actually recoup from generating something out of it. Right. It's the the hard logic of uh building something with a high fixed cost is that you must believe you're going to be doing enough deployment to justify that. And so uh you know I think that was sort of was was part one and then this bet that we would have you know what we call kind of test time compute scaling the advent of reasoning models they didn't arrive until 2 years later for which I'm you know very grateful to uh Nome Brown and
43:52the team at OpenAI that was the first to kind of you know publish one of these um uh I think what's happening today is uh we continue to see this sort of vecture of these test time compute kind of scaling laws, right? It's this stack of different ways that we pursue greater capability. So, one is, you know, 10xing and 10xing the amount of compute we use to train these models. Um, but the other today is how do we how do we pour more
44:22compute into deploying these models into the inference side and it's a a huge part of how you get to greater capability. You know, if you look at again the uh uh examples of solving a Millennium Prize problem that Navia Stokes uh uh solution was, I think 80 hours of execution from 10,000 agents in a swarm. And so you have there these kind of two axes that trace out this enormous kind of volume of token
44:52generation, right? where I've both got 10,000 parallel agents all generating their own chains of thought and then they're running for 80 hours. Um, and so it is so different to the kind of intuition that we might have about an LLM where we sort of think we ask a question and the singlethreaded thing will go and generate almost immediately an answer. when we bring these models to bear on the hardest problems and I think
45:21generally the hardest problems and this is a bit of good news you know are also the ones that have by far the best you know the most positive societal impact it generally it is solving a science problem or uh you know something in medicine that we maybe haven't been able to solve before um to go and do this where you have this incredibly economically and society beneficial thing to go solve which is beyond the reach of kind of
45:48just you know applying the model uh and asking it one shot you know what do you think what's the answers to this inference compute is the access that we have and the Navia Stokes example is is is kind of evidence of that and so I think the thing that is what do you mean inference compute is the access that we have is the access that we have to it's the it's the it's the knob that we have to turn to get greater capability so if you imagine that at at any moment in time we have in the world the best
46:17frontier model. It will come from one or other lab at any point in time. And the best frontier model is extraordinarily capable. You talk to it and it's smarter than any you've spoken to before. But it turns out that when you ask it, you know, hi, I'm working on this, you know, I'm working on this challenge. We're really trying to find um, you know, a a cure for this rare disease. We're not at a place and of course we're not at a place where it's just going to say oh here's here it is like I've thought of
46:45this it's you know in in my neurons this is uh fired out immediately but the thing that we also discover is if you do throw enough compute at it if you say hey how about rather than just trying to give me an answer you propose you know many hundreds of candidate exp you know potential solutions that you could go and explore and you create multiple copies of yourself and you go and traverse the internet and you read
47:13the literature and you debate with copies of yourself and you exchange information and come back to me in you know basically a week's time once you've generated maybe billions of tokens in pursuit of finding an answer you know then a little bit like if we gave a bunch of smart people a lot of time to solve a problem you do have a chance that you've come back with a solution and and we really see this this is the key it's what every single single application of a frontier model to solve
47:41a hard problem is going to look like.
47:44My question to you is I I am a a venture capitalist. I'm a investor and I see the crunch, the debt cycles, the credit, the problems that we are hitting now and about to hit. There is not enough money in the world for the requirements of this cycle and everything that you just said there which I agree with. And then I look and go and then we haven't addressed the physical world and physical data and robotics and everything in between that's required
48:14there. And so I'm holding these two seemingly opposing thoughts in my mind going what the happens now, right? Yeah. Well, this is why we need continuous innovation in every part of the stack that leads to serving intelligence, right? And the good news is, you know, for all that obviously we're focused over here on chips, you know, if you just look at the algorithmic side, like model efficiency, the cost assuming no hardware change to serve a given level of intelligence is
48:44Exactly. Improvements in efficiency with no hardware change.
48:47Yeah. And as we see, you know, longer context becoming a challenge, we see incredible innovations on how you can compress context and do sparse attention and other ways of handling this. So there is sustained innovation on what you might call kind of uh the compute efficiency of the models you know at with the same chip at ISO compute you know how much more intelligence how many more solutions can I can I arrive at and then of course we do have the same on hardware and we're working very hard to
49:16make sure that we have that hardware effect as well and so I think what's what's really thrilling and it's the reason that I think on the whole you know this buildout is entirely justified and we won't see you know a a collapse in this market is that you know far from the uh fiber buildout of 20201 you we have complete full utilization of the GPUs that are powered on in data centers we don't have dark GPU fleets
49:45because as you've seen you know we have this enormous number of problems as a species that we still have to solve we now have tools that we can throw at those problems Um, and we have many orders of magnitude more thinking that we could be doing at sort of inference time to solve the hardest challenges if we had the compute. And so, you know, I agree with you on the one hand, it feels like we need thousands of times more compute than we have today. We're not
50:15going to get thousands of times more uh data centers and I think that's probably a good thing for the world. What we will be able to do is take, you know, roughly that kind of land and power and set of data center sites that exist today and over the next few years through both model innovations and hardware innovations coming together squeeze more and more and more dramatically more intelligence per megawatt per rack in one of those data centers. So I had Mike Cannon Bruce on the show from Atlassian
50:44and um he said to me uh you know I think the cost of inference is going to come down dramatically over the next 3 to 5 years and that's why you that's why I'm not an actor. Um you should do me in the next one.
50:57Uh well I mean you in the next one is relatively close to just me.
51:02Uh you want me doing Russian? That really is a good one. But with the Mike Cannon Brook theme in mind, um what happens to the cost of inference?
51:12So I think there is you if you look at the cost, what what will a token cost in 10 years time? Um it's one of those sort of questions that you know begs another question, right? Like what is the model that we're running? It's probably orders of magnitude larger than the model that we have today. And so that token is also delivering in some ways you know substantially more capability. Um I think the margins in inference is
51:42something that will you know will fluctuate and they will occur in different places. We will be spending far more on running models in 10 years time than we are today. You know that part I think is is clear. Um but who is going to win in this space? you know, you've got today, I guess, you've got chips, you've got clouds, you've got frontier labs building frontier models, you've got folks that are building and serving open models. Um, and I think one of the reasons why, you know, I am very
52:10long on chips in terms of where that value occurs or being one big place where that value occurs is actually because of the nature of the challenge of placing the right chip bet. you know, it's it's got this intrinsic kind of lag. That lag to design the chip, to get the chip out, to have it built, and then the degree to which it has to have a a useful lifetime in a data center, uh, means that you don't have quite that same kind of fully efficient race, you know, potentially a race to the bottom
52:39in some ways, uh, in the chip space as you do elsewhere. If you have a better idea, if you can build some sustained advantage in some way, um I think you see a lot of the value acrew there. Um I think you also see a lot of it acrew still at this frontier. You know, the construction of frontier models which have this sort of sustained advantage over some of the others. It's a costly process. It requires incredibly talented teams working with huge amounts of
53:07compute. Um but it is something which then delivers you know who doesn't want the greatest intelligence on earth of course people will continue to to pay for that and so I think the overall kind of prize pool expands uh there will be kind of relative winners and losers within that but it is a it's a positive sum game in inference we just had Sat on the show um the other day and he said that we would see a future where every company has their own specialized model with their own unique and proprietary
53:36data and actually a cool kind of value segment would be eval benchmarks for themselves and for their company and for their own objectives.
53:44Do you agree with that world of many specialized models on a per company basis? Do you think it's it's relatively technoptimist and wrong?
53:52How do you think about that?
53:55I think there's absolutely space today and just if you look at the technology there is positive impact in taking a model and fine-tuning it for your use cases. It's a transient thing, right?
54:08Like generally the advantage of having something that is, you know, internally fine-tuned will disappear as soon as the next release of the next open model arrives. I think that can still become a bit of a kind of yes and right like as long as you have a model where this as in a you know a model of constructing these where this stacks on top of the best frontier capability um then I can absolutely believe that there is you know an advantage to having a kind of an internal a specialized
54:37variant of something that is otherwise constructed at the frontier. Um, but I do think this is one of the the hardest questions to answer. Um, because over time it may well be that, you know, you can only do this with the open models, for example. Uh, and if that's the case, um, you know, what does it mean for, uh, you know, this sort of trade-off where is it better for me to specialize an open model or just to go all in on the thing that's been released, you know, at the frontier that is still pursuing
55:06these better compute efficiencies and by extension might actually be very cost effective for my challenge.
55:12Totally get you. So when you look at that value stack, you said obviously chips, no where you think is the most valuable, where is the least valuable segment?
55:24I think there is a danger for anybody that is working, you know, purely in the software layer. Um, I think, you know, I've seen a few uh plays recently on kind of orchestration.
55:39You know, there's a coming wave of heterogeneity in the chips that we're going to have in in the world. you know GPUs, other XPUs, these new chip efforts um and you know some clouds and some software providers have set themselves up to be the kind of orchestration layer over uh over this sort of coming wave and I think there's a world in which that kind of heterogeneity is is a slightly transient thing you
56:06know that um yes today you know we have for example uh a world where Nvidia having acquired Grock will deploy two chips alongside one another, you know, Reuben GPUs and Grock chips and they'll do what's called prefill decode disagregation and attention disagregation where these pieces of silicon are doing different parts of the LLM inference workload. And so there's this sort of sophisticated orchestration between at the same time it's kind of like model rooting but chip rooting.
56:34Yeah, exactly. Chip routting. And you know, I think there's a kind of um kind of uh technologist view that maybe hey why stop there? why not have like seven different pieces of silicon and maybe they're from different vendors and they're each specialized in different parts of this. Um, and I think there's a belief that this might be a a huge kind of wave of value creation will be in being specialist in how you orchestrate over these. I think a lot of that could collapse back down over time. you know
57:02that that there isn't a fundamental existential reason why we desperately need to have seven different platforms to run a workload. Um and there isn't really a strong economic case for kind of why we would be able to sustain that as well. I mean if you imagine even just the the Nvidia Grock uh data center deployment, how many Nvidia GPU chips do I put in there versus Grock chips? you know the the bet today would be so this
57:30attention feed forward disagregation uh if you'll give me a minute on the tech here I will is a uh you know I put the weights of my model on the gro chip because I access them really fast on that high bandwidth and because every user that I'm cramming through that data center uh they all want the same model weights right so I can kind of um uh amatize I can share the cost of that much more expensive memory I can recoup it make it up on volume attention which is the part that operates over the
57:58context uh isn't shared across users. My context is different to your context.
58:04And so for that piece, I'm incredibly exposed to the cost of my memory. And we talked earlier about how SRAMM is far too expensive. DRAM is much cheaper. So I need to fall back to the GPU and the GPU's memory to run that piece. And so that's the logic of you know an Nvidia data center with these two chips in today. But as I spec chips do I put? You know what if my context lengths get longer over time?
58:27Suddenly I probably wanted to have more Nvidia GPUs than I did last year and the context lengths have got longer. How much longer will they get? Maybe the model gets larger suddenly and actually I wish I had more Grock chips. Um and so you're very likely to end up in a place where some portion of that capacity in your data center is stranded. You're just not filling it because you've got the wrong mix. Um that's unacceptable and it's completely unacceptable, right?
58:52Because you have a certain number of megawatts. you're trying to juice as much revenue as you possibly can out of them. And so this idea that you're pa placing a fixed blend and you're not going to be able to change that data center for you know the next few years right that's fixed it's capex it's being amotized over time heterogeneity is is actually a really challenging thing and I think it's a transient response to the fact that you don't today have a chip that can kind of do it all right these SRAM based chips solve this fast inference challenge a bit right but they don't have the memory capacity to do
59:22attention and so we still need to bring that GPU in they also don't have the flops density back to the top trumps to run the compute inensive part of LLM inference the prefill portion the prompt processing because uh you've got this one piece of silicon which cost you a lot and you've had to share the area that you might put maths in with the area that you put memory in. So you definitely haven't got as much maths as your counterparty that used HBM who
59:49could use 100% of that chip area for the tensor cores that run matrix multiplies.
59:54If you look at let's say what Fractile is doing, you suck the SRAM upstairs basically and you have a you have a DRAM layer stacked on top. All of that compute die on the base is now free to put tensor cores down. You can have as many flops as anybody. You can be at top trumps maxer on that. But you also get the SRAM level bandwidth from the 3D access and you get the capacity from the DRAM. And so suddenly you have this kind of you know holy trinity where you've you've got all of these things in one
1:00:22and it becomes clear that you don't need heterogeneity and all of this complex orchestration and running the risk of stranded capacity.
1:00:30Was Nvidia's acquisition of Grock and insurance by I think there was a there's so many fun rumors about this right like I think the Grock team is an incredibly smart team and Jensen is a smart guy that wants smart people on his team. Um uh they also did build a platform that I think answers or answered a a big challenge that you know Nvidia faced at the time which is how do we move into the fast inference space. Um I think you know I
1:00:58think Jensen is quite positive some in this space. Um he said some nice things about Fractile last week when he was in London. Um you know and and is in some ways a bit of an ecosystem player. Um the way that Nvidia, you know, captures the kind of bulk of this market is that, you know, they will sometimes be deployed next to other platforms, other platforms that augment parts of their capability. Um there is another rumor that obviously Cerebrus got to deal with
1:01:26OpenAI. Um, I think that can be an alarming moment if you're running a multi-trillion dollar GPU company and one of your largest customers suddenly signs on for something that you perhaps hadn't previously thought was existentially necessary, which is a fast inference platform, and what's your response? And I think both deals were signed on the 24th of December, uh, which must have been a fun time for to be a lawyer as well. And so it's being a lawyer is a brilliant time right now. Uh, and so you think it's a knee-jerk reaction to that
1:01:55diversification from open AAI?
1:01:57I think it's a acknowledgement that some parts of this space at the time required a different solution and this SRAMM play from Grock was the right solution at that moment to be able to run alms faster. Today though we've already seen ourselves I think move on from this. You know you cannot solve LLM inference with these SRAM based chips. There isn't enough capacity. uh the cost to achieve speed is too high. The constraint on the context lengths you can operate on is
1:02:27very very stark. And so you know I'd sort of characterize that solution as uh you know it gives you sort of snappier chat bots but it doesn't necessarily give you this thing of being able to take kind of overwhelming frontier intelligence these agents that run for days on end and truncating that to be orders of magnitude faster. I think that really is where the huge prize is. Um and today that's not something that that provides an answer to.
1:02:53So um the different players in the inference space that people know are your modals, your fireworks or your base 10s, your togethers.
1:03:01You bet on one. Which one would you bet on?
1:03:05I'm friends with founders of multiple of those companies. So uh children, you always have a favorite.
1:03:11You know what I I like about all of these guys is they still have the capacity to dodge and weave, right? they they haven't yet locked up huge amounts of their own capacity. They're not a neocloud. Uh they have the potential to bet on a new chip as it comes up. They have the potential to carve out new portions of the market. I think there's a world in which this piece doesn't fully commoditize, right? You can be specialist in serving multimodal models,
1:03:39text models, fast inference. Um so I won't place a specific bet. I will say that I would rather be an inference provider today, you know, a specialist in running these models, getting them up, um, than I would a neocloud. Actually, I think there is more mileage in being able to create the sort of business flywheel of of of signing up these enterprise customers that consume these tokens.
1:04:07Will they be multiundred billion dollar companies? And is it an Uber lift market where winner and significantly less valuable second and third place or is it a GCP as your AWS market even that's only three players I guess.
1:04:22Um so yeah I think you're not going to have 10. Um and today you probably do have a long tail that takes you towards that. Uh and if you add the neoclouds into that you definitely have you know tens of players all of whom are maneuvering around billions of dollars.
1:04:38Are they multiundred billion space?
1:04:40I think we'll have a couple of multiundred billion companies built in that space.
1:04:44Can you talk to me about the new funding round? How much did we raise?
1:04:48Uh it was a $720 million series C.
1:04:52How was the process? Was it fun? Cuz now you're like the bell of the ball.
1:04:56It's um you know, if you go back four years, actually go back two years, you know, we were still teaching people about the word inference. And it's it makes for a it makes for a kind of trickier investor pitch, right? Because uh when you educate an investor, it's just it's kind of like leave then and you've got 30 minutes, right? And uh you have to spend 20 minutes. Uh four years ago, you had to spend 10 minutes explaining what an LLM was. And then you had to spend another 10 explaining why you felt that training these things
1:05:25wasn't going to be the be all and end all. And and you'd deploy. I'd then spend five saying and here's why I think speed is going to matter. we hadn't yet seen reasoning models and so on. Uh so compared to then you know it's been it's been very fun I think to to be able to go straight in and talk about how we have a completely unique and extraordinary team that can do things differently to everybody else. How we have a very unique position in the supply chain and can maneuver differently. I think a different set of
1:05:54philosophies on how you build to win in this space. You know that's what I get to spend my time over coffee talking about. Um, and so I think there is a real advantage to now being in the eye of the storm in a way in a space where I think there is an appreciation of why this is such a crucial problem to go and solve.
1:06:18Talk to me about is that enough and suddenly you have that in your account.
1:06:24What changes like what do you do that you couldn't do before?
1:06:27So suddenly there's a there's a weight of responsibility, right? like there's a number of folks who have uh you know uh staked a lot of capital on us and I think one of our responsibilities there is to ensure that we are always have you know putting multiple shots on goal as it were. So one thing that capital allows us to do is kind of distribute ourselves to look at multiple things at the same time. You know we're now able to start looking more generations ahead.
1:06:53We're able to explore more parts of the stack than we maybe maybe previously would have. you know, more innovation at the systems layer, at the networking layer. Um, uh, and sort of building this entire road map of of future systems is one thing that, you know, capital allows us to do. Um, this is an expensive space, but I think it's a space where, you know, the underlying capital that is needed to to kind of get that chip out, get those first systems back, you know, it's that's been a kind of fixed cost
1:07:23for the last few years. And in some sense um you know that's not what most of this capital is used for. I think what a lot of this is now important for is is that ability to maneuver faster in the rest of the world in the supply chain. Uh you know the placing of long-term agreements over certain components that we need to have years into the future. um uh and the ability I think to build you know more of that
1:07:51muscle of predictive power as well. You know we can now start to think about how we run more of our own experiments work out what might be important in the future and as I said Fractile will always be a chip company. Um, but we do need to have strong insights so that we can be more helpful to our customers about what scales well, where model architectures might head next, how you can handle very long contexts, what it
1:08:17looks like to run an agent swarm versus one very long serial agent, you know, all of these almost ML research challenges that we also get to now go and tackle. Um, and we're also expanding geographically. So, obviously we've got folks in Taiwan. Uh we've got a team in the states. Um I'm very excited that we were joined just a few weeks ago by Dan Rabinovich from Meta uh who worked on a lot of their AI compute systems there driving our kind of systems buildout. So
1:08:46everything from the chip upwards, you know, the rack scale systems, the networking, the integration with the data center. Um so it is this very long stack and we get to go and build parts of the business, you know, where it makes sense to. um and pull in the world's best talent as we as we go after that as well.
1:09:07Can I ask you speaking of investors, if you could invest in any one of your suppliers, who would it be and why?
1:09:13So TSMC is uh kind of an extraordinary business in that if you look at everything that is happening today, everything traces back to them, right?
1:09:25And it's sort of I mean this is not a this is not a scoop for your podcast, right? I think it's pretty well known that TSMC underpins all of the world's chip production. Uh it is cheap at the price. I mean I I would buy that stock every day of the week. Uh all of the world's chips that are at the frontier or the majority of the world's chips at the frontier are built there. Um they are an extraordinary business, an extraordinary team. Um so I think we'd we'd go in there.
1:09:51I'll take that. If you could invest in any of your customers, who would it be and why? So, I really like some of these scrappy Neolabs. Again, I'm thinking like an investor here. That's what I want. They're a little cheaper. Um, and I think we're going to see some Which which one?
1:10:09Uh, so uh well, friends of mine have started inherent in London. I don't know if you've come across them. They're looking at uh uh both I guess the sort of the RSI challenge but also in particular what it takes to build systems that automate science uh automate research. Um, and I think there is a lot of mileage here in, you know, it's not quite the same as pursuing the best frontier model that goes and tackles everything. But I think there are so many interesting questions in
1:10:39what it looks like to orchestrate and deploy models, build systems that are better at at solving these challenges.
1:10:46If you were forced to invest in one of your competitors, final one, before we do a quick fire, who would it be and why?
1:10:52So, I'm a big believer in 3D RAM. Uh and one other company called Dmatrix is pursuing a different architectural direction in the same space. Um uh I think both companies are going to be very successful.
1:11:03What was their last prize?
1:11:06Um okay. Uh are you ready for a quick fire?
1:11:09So what have you changed your mind on in the last 12 months?
1:11:12Maybe I'm not quite ready for a quick fire. Do you want to read me the quick fire? Sorry. And I'll just uh Is that Can we keep that in? um a no one's ever ready for the quickfire but um so one is what if you changed your mind on the last 12 months I think the underlying pace of capability advance in models has been extraordinary so these math results that we saw this summer uh in terms of kind of you know people talk about the truncation of their timelines right I
1:11:42think what has become clear is anything that is kind of verifiable you know maths and coding tends to fall into this uh will be solved in the next 12 months we will be able to tackle any of these kind of hard verifiable challenges with frontier models.
1:11:57Who do you not have on your board that you would love to have on your board?
1:12:00This is an easy one. I think Deis Saris is both an academic hero of mine and a bit of a British hero. Uh and I think he should uh he should come aboard.
1:12:10Have you asked him?
1:12:12Well, this is a good way to do it. Um Deus, what's the biggest misnomer about the chip industry today? I guess that it's a you know misnomers about technology businesses in general are that they're very technological and it's a very very very human space. So much of it is about relationships. So much of it in our case is about having this extraordinary team that can collaborate across the entire stack. Um so it's that it's powered by humans.
1:12:38Will Nvidia be a 10 trillion business in 5 years?
1:12:42You said that pretty confidently.
1:12:45It's a very positive sum game. Nvidia today trades uh this is a sound bite so it may be unscored but I believe it trades at a lower forward multiple than Cisco after the crash uh you know after the com bubble burst. Um I'm very long on this space.
1:13:02We've talked about this bubble before it even started booming, right? We're so traumatized by the dot bubble um that we've priced in exuberance right from the outset. Um, and so, you know, I think we're not yet appreciating how sustained this buildout will be. And you don't have to have incremental data centers added necessarily. You know, we talked about this advance in compute efficiency. If I have finite numbers of
1:13:30megawws, I need to replace my chips every few years with better chips. And so that's a huge advantage for Fractile as a market entrant because every chip that you see in a data center today will definitely be out of those data centers, you know, in six or seven years tops.
1:13:48Many of them won't last more than a couple of years. Um, and so that's an enormous market opportunity for the market leader. They have extraordinary supply chain relationships. They're going to do very well. There is then this enormous and I would say new opportunity in being the chip vendor that serves the needs of the frontier and so I'm very long on Fractile's chances over that horizon as well.
1:14:13You are in charge of the UK.
1:14:15Congratulations. Um what do you change to make the UK a superpower in technology?
1:14:21I think you know sadly so much of what unlocks growth is unlocking energy. It is doing everything that we can to bring online more renewables, more nuclear, more sources of power that allow us to have build out of industries that matter here. Final one. What are you most excited for about the next 10 years?
1:14:44I think it's again we're using these models now. What are we using them for?
1:14:48We're using them to solve problems we couldn't solve before. And I think we're only just starting to see that pick up and take off in terms of what it means for the real world. And so it is going to be all of the ways in which life gets a bit better. Uh we find you know cures for diseases we couldn't previously tackle. We have education that is highquality for for everybody. These things are unequivocally going to come.
1:15:15We have the technology to make it happen. Now I think it's so awesome.
1:15:18Like you look at like class sizes traditionally like 40 children in a class. With AI, you have like personalized education for some that like multiple choice, some that like visual learning at their own speed. I was at math, probably unlike you. I was always too scared to put my hand up and say, "I don't understand cuz I'd look stupid." And like having personalized learning to to that point is like one tiny example. I so agree with you. Listen, Mort, I love doing the show cuz I I really get to learn from amazing people like you. So, thank you for doing it with me and this has been so much fun.
1:15:46Thanks very much, Harry. It's been very fun.