0:00I actually don't know what we look like, but okay.
0:03Hi, Jerry. Welcome to Venture with Grace.
0:06Thank you very much. Very happy to be here.
0:08Um, so maybe we can talk about our Halloween special costume. [laughter] Today is Halloween.
0:14Today's Halloween. Okay. You want to give the rundown of like why you pick the Witcher?
0:19I'm a Witcher. Geralt of Rivia, Slayer of Monsters and Transformers.
0:23Yeah, I'm the Yeah, that's the person I am on right now. There's a specific architecture of neural network that I think got its place in everyone's homes and AI assistants and we need to slay it and replace it with something better. [laughter] I really like that you can tie anything into AI these days.
0:42Um I want to start with Okay, let's talk about like core automation. Obviously like the big thing is like in the past everybody's talking about transformers and then you guys are kind of the transformer slayer. Um tell us more.
0:56um like PE transformers have been so successful that the AI researchers kind of I think became complacent and start believing transformer is the end of the of the road. we've been able to scale those multiple orders of magnitude and we've seen improvements and we've seen new skills and I would like to be a little bit counter culture to that and show there are actually different ways how you can arrange your neurons different ways how you can arrange your memory on a GPU that will yield itself
1:24to smarter models and better more robust learning systems of the future and this is what we are to do I'm very happy in some what transformers have already delivered I'm very happy of the AI assistants that that that are there today but there are there is a better technology to be built and this is what I together with my company have have set out to do.
1:44Um I have so many questions. Number one is I feel like transformers is kind of like the specialization is kind of for like you know um identifying which part of the context or content is like more important. So that's kind of like the specialization and then um it also takes up a lot of like training data and stuff like that. So for you guys it sounds like you guys don't require a lot of data.
2:08We do require data. Data is necessary.
2:10We think we can be more data efficient.
2:12There's some multiplier but data is needed. And I think I think the biggest thing what we want to get rid of is like in some way product specific but even more importantly customer specific fine-tuning. Currently the big labs are purchasing huge amounts of data for how to make models good at health, how to make them good for lawyers, how to make them good for programmers, how to make them good for making valuation of companies.
2:39And I think it's unrealistic for the models to have like knowledge of in in one place of every profession and everything what we want them to do.
2:48What's what we at core automation want to do? We want to build model that learn from the user data and on the fly working with the user. So if you are a law firm, you don't need to necessarily buy offtheshelf model that knows everything about your your company's cases but can learn together with you as you bring it along and show it to the data that you actually care about and you wanted to do them all improves from that. So like move more of the learning from the lab into actual like customers
3:18hands. M wait um say more. So basically I feel like nowadays like the mo most amount of people coming out of our podcast talking about like open weight models. So essentially you um download stuff and then like running stuff in your within your own like database. But um how does how do you guys like kind of like different from that?
3:38First and foremost we use different algorithms. The the problem with open weight models is that fine-tuning is hard inefficient. that requires a lot of people and the results are not really guaranteed and not really very good.
3:49So you guarantee results.
3:50Yes. And we want to like through a different architecture we can provide a lot of the things are just hard to do with finetuning transformers like on one hand you have a neural network on the other you have data and there's a question how do you combine those things together? fine-tuning is one way to do it which like arguably is just not very good and like we are building alternative to that that we think will be will be much
4:17much better. M um okay so I want to talk about which part do you want to talk about more like the people or the business logic cuz I feel like obviously let's talk about the people because like you know you guys obviously hire like the chief people officer like the former chief people officer of open eye who like obviously have been at Hudson trading for over 15 years who have like I'm sure like she have like recruited like multiple like a ton of like amazing
4:46talents and then last time I tracked back on uh your team is like uh you know like 16 20 people something like 27 at this point.
4:54Okay. Wow. 27 since like the last time internet has updated. Um so I have so many questions in the AI talent space. So obviously one of the things is like uh talent nowadays like worth a lot more than the past. Um but nowadays like so many decisions are actually made by AI like why is some people more valuable than others and when you guys are thinking about talent planning what like what is considered
5:23like a good talent? So you come from the training era or like trading not era but like the trading um squad. So what is why is that like a better significant better like AI people than you know people who is like you know studying computer science.
5:41Like we definitely care a lot about talent in many ways, right? Like when you are starting an AI lab, talent is your one of your most important resources. And um the the type of team we want to build is generally a small team of people that can move very quickly and build some of the frontier frontier systems. We we enable people very deeply with automation with agents. So we can we can can build a lot with a pretty small team but there's still a number of people
6:09that you need and sometimes I am joking that for today's models like people serve as continually learning meet proxies on top of them and we obviously like you know Julia has been a close friend for a while and we basically like decided to start this company together and I cared about how do we build a team and a culture and how we work. This is this is what we are doing and I think we are picking every person like very specifically to fulfill a specific role.
6:34We are not just just blanket hiring people that we think are are nice and smart. We have a mission to do and we are building a team almost like you're drafting a football team. This person in this in this role, this person in this role and that's that's how we are we are operating. We are thinking that companies of the future generally will work very differently that companies of today with increased automation of AI being able to do a lot of a lot of mundane work and a lot of grinding that was pretty labor intensive in the in the
7:03times of the past. The question is how many maximally enabled humans do you need and how to structure the work of humans that can work much faster than than ever before into a company that actually delivers a pretty audacious and a pretty pretty difficult mission.
7:20You mentioned like the talent or like the company structure will be different.
7:25How different also like are you tired with like the two sword in the back?
7:28You need to have two swords. [laughter] One is for monsters just in case.
7:33Okay. So the um how is a company in the future different from today?
7:37Yeah, like I I think like company of the future surely is largely AI operated.
7:43AIS will be processing all the information flowing through the company. AI will be proposing all the decision making. AI will be operating a lot lots of software and and machinery. There are there are questions kind of where where where do the humans come to in that process? And the the future that I like the most is where the humans generally give the objectives of what do we want to achieve with the company's resources. We should take advice from it. But I like the world where we are clearly in control
8:12and in the steering wheel of the increasingly automated systems that can generally fulfill what we what we want to do. And that's kind of like I see there's a potential future where the whole company is just the board meeting where there is a decision how do we allocate company resources? what do we want to do with those resources and then AI goes and does it and there's a next meeting where we see how did that go and what's the what's the next thing to do that's that's you know still will take a
8:39few iterations to get there but I I do see how like slowly but surely we are we are inching towards that okay so another question I had was like you know nowadays like I'm personally using you know you name it a lab I'm probably using their product So besid obviously you guys since like you guys are kind of stealthy um what do you product out yet?
9:05Okay. So what do you guys use inhouse like do you have your own like [snorts] everything like your own building more and more stuff ourselves. So we have internal code or no but we we we have started using codec and we've been using it for a while. We are slowly migrating to to our own solutions in some ways like the state-of-the-art models are very good but the layer on top of models still has a lot to improve and a lot of
9:32like how do we use AIS like I think I think well evolve and surely does evolve codex is kind of like a very general tool that can be useful for everyone but actually doesn't add that much value on top of just just the model we've built things that we think are both more flexible give individual more power but also like tailor slightly more to what we are trying to achieve and how do we want to work together for for for us but the the models we are we are using largely are still are still open
10:01AI models um what are something that you feel like you would take away whether it's like you know culture or how things have been done at open eye that you would take to your own company and then what are some like you know the new things that you guys kind of have been implementing the thing I care about the most and I brought with myself when I when I started this new company is being the innovator and daring open AI for a while was a company that had a very audacious dreams of building artificial general
10:30intelligence it didn't have that much idea what's how to do it but it had a lot of conviction and a lot of confidence in going [clears throat] in that mission and for many years openai did breakthrough research like year after year going towards that goal and building very being very strategic about how to build compute towards that goal.
10:50How to build a team towards that goal and was going there like we wouldn't have AI that we have today without without open AI being there and I think all of us who think open AI is a AI is a positive force in the world we should be we should be extremely grateful for that. So like daring high having having high ambition and um actually focusing your forces on innovative research is what I what I what I want to really uh take into that account and just just because times are different and plays
11:20are different. I don't think a company like OpenAI is possible even today it was possible in the day it started but the landscape has shifted a little bit.
11:28I'm trying to take like the best of the research culture open of open AAI and trying to add a little bit of more of our own like the place where we are which like things move very quickly so we need more sense of urgency these days. We are a bit more of a counter culture and underdog because we are betting against the the existing paradigm and we obviously are much smaller and have less resources. So we need to be much much scrappier than than companies of the past. I mean you have 52
11:56no 532 million for seed that that feels very small compared to the budget of what's where do people spend the budget and also I I believe in Nvidia is your investor as well. So like does it just go straight back to it goes straight back to Nvidia all we [laughter] want to buy the best GPUs on the market and we want to buy a lot of them.
12:19Okay. So just straight up GPUs.
12:21Yes. spending money on anything else.
12:25We spend a little on the office but not that much. [laughter] Okay.
12:28And snacks. And the we have pretty weird and fancy snacks.
12:34Um salty salmon skin. [laughter] Salty salmon skin. At some It's popular in Asia. I haven't had it before, but some we had an agent that automated ordering snacks and that distribution was getting weirder and weirder all over.
12:52Wow. It's a it's a very important task to hand it to the agents.
12:56Most importantly, it's not something you really want humans to be spending a lot of time on.
13:01Um what about like the business model? So like I feel like you know open AI and I guess like there's two things. One is like one is like the strategy of the company and then the other part is like the um business models and things. I feel like nowadays like um Oh god. So sorry. Apparently whenever we're recording people start making coffee. It's very important for people.
13:27Yeah. A lot of people care about coffee.
13:29Um I guess like when you're thinking about the business model or like you know the past like the open AI um and anthropic situations like I guess open eye like started with like consumer driven product um I like I mean from the news it seems like it was not supposed to be like as famous or like as like popular as it gets but it became like a very popular consumer product and then uh on the other hand like anthropic went to you know um building like a very strong like work tool
13:59And I'm just curious like from the business perspective like how do you think about positioning yourself like in the business arena and then since you guys are I guess like you guys are um I was like chatting with a lot of executives from like different um AI companies like people were saying this is like the last badge of um you know foundational model getting funding. Um what's your kind of like opinion on that? Is there more like you know AI labs gonna come around and why or why
14:29Yeah, I I do think we may be one of the last batch of companies getting founded mostly because of the runaway effects of AI and models being so good that it will be hard for the for the startups to be able to to to to like catch up and enter enter the ring. I I I do think that a lot of reasons why startups exist and can win is because larger companies are inefficient and larger companies are moving too slow and cannot cannot adapt
14:59in a place where where a nimble player can in a in a rapidly changing landscape. What AI will do in the next few years is that the big companies will be able to move very quickly because they they will not have to have all the political uh issues around them. they will they will just be able to move AI to direct resources. So I I I do think like that will
15:23I think we can keep going. So sorry I do whenever yeah like this okay so we're talking about like um there it needs a lot of resources for um keep going sorry yeah so so the story was I think the AIdriven efficiency will likely make big companies be able to like compete much much faster and in some way if there is a future without work and without labor
15:52it's hard to imagine how like starting up companies exactly will look like.
15:56There's probably some amount of risk takingaking that that may may may take place but I think I think I think it will be it will be different and and it's it's hard exactly to predict how the future how the future will look like there but yeah what we what we are largely focusing on for like our models and what we are building is models that do work for you. We core automation is not interested in building your friends not your your therapist. It's it's like the model that is meant to like you tell it what you want it to do and it gets it
16:26done for you generating value on the tasks that you care about and all the all the work you want to be done and we don't even want to build the malls that have that ability off the shelf that that know everything you want you teach them you like like on boarding an employee you take that model and show them this is this is how my house look like this is how my computer looks like this is how my data looks like these are the things I want you to do I will show you a few times and then I expect you to be doing them for me.
16:53Would you say like core automation is going to become like a agent that's like an agent replacement?
16:58This is meant to be an agent, but it's I think it will be a slightly different experience with an agent. I think it'll be a bit more personal if the agent meaningfully adapts to you over time.
17:08Do you like my a like my email that I'm sent by instinct?
17:12No. [laughter] Was it personal enough or no?
17:15I haven't paid attention.
17:17Okay. So I'm just curious like what is the difference between like the um you know the current agent landscape of um you know nowadays like everyone have like five agents whether it's like rockbot or like you know name your favorite agent um what's a core difference between them and then um and then core automation.
17:34Yeah. Yeah. Yeah. I I think one thing how the all of the personalization of the current agents happens in context.
17:40So you have usually have a system message which you can you can can specify a little bit and then you can put it some memory files that the mall sometimes sometimes can use and this is like this is great this is how the today's systems work but one thing the agents don't really learn any new skills in a in any meaningful way like if there is something that agent has a hard time doing now it will will not improve itself until anthropic or open AI releases a new model for you or or Kimmy
18:08or for for for for for the sake of completeness and like it's has a hard time getting things that are not very easy to verbalize like everyone who has ever prompted model like knows and understands that not everything is very easy to describe like the system messages you can try to steer the model in a particular way but very it's it's much more often frustrating than not sometimes you think oh maybe I could show them all maybe I will tell them hey a little bit of less of this and a
18:38little bit more of But um such an update should be happening and again in a in a model that learns from you. It will will just just happen. The model will work with you and within the next day will get what you need in your context slightly more and slightly better and slightly slightly understand what you are what you are asking even if you are vague and incomplete better that the current models like that just they have very limit ability to update themselves.
19:04M so when the model is updating right now it's based off like the data it feeds into it and nowadays like you know each company is feeding their own data into the model um so I guess like what is the format going to be like significantly change in the core automation agent era and is there any particular like use case that you're imagining this like going to create like a significant differences the the thing what I think about the
19:33most is thinking about how companies operate.
19:36How do we take companies to a place where humans come in and do their work?
19:43Maybe with a little bit of help of agents here and there. But you are hearing more and more how a lot of companies are having even a hard time finding the places where the agents are meaningfully useful and how to turn those into productivity into places that are like largely driven by AI work and agentic workforce and places where we where we kind of like think of it as less as a places of labor and more like places of resource allocation.
20:10H um I guess like when it comes to like charging so nowadays like people are doing like usage based charging um things and then like some people are kind of like replacing the human um of like different workforce. Um, do you feel like that aspect will change and for for people like you know listening to our show or like you know for a lot of our LinkedIn audience sometimes people are looking for jobs in the AI industry and um what has that meant for them like are people going to be like
20:38doing something significantly different or you know um obviously are um yeah like people could be cooking or [laughter] doing other I do believe we are heading towards what I call post labor society.
20:52Okay. There is a future where we don't necessarily the water be like yeah for example like I I sometimes imagine life like like high school where where high school is a moment in life where no one has to really work you don't have to generate value you mostly spend your time learning and hanging out with friends and if I if I imagine like how could a life look like if I was completely free of any obligations I would a much better high school experience than me I was doing homeworks
21:21[laughter] You can still do homework if you would like that.
21:25Um, I was definitely somewhat somewhat of an anti homework person.
21:30You're anti homework person.
21:32Yeah, I don't like homework that much.
21:34Yeah, you look like you will finish a homework within 2 minutes [laughter] and then people will copy your homework.
21:38I would usually in the morning but but yeah, I I don't like spending my my time at home on it. Um so what do people like so what like what kind of job will be replaced the fastest? It's it's hard to say really. I think I think it will more be like various types of job will be replaced gradually like when we look at the job market today was the job of programmers really replaced I don't think so but it was already radically
22:07transformed every every simple engine every single engineer can write code much faster and much differently than before. there are whole groups of people that stopped reading code completely almost in many ways. So the profession has changed the dynamics of it has changed and I think that also affects like job market but I don't think we can say anything got replaced.
22:30I think I think like the replacement will happen if anything like all at the same time rather than rather than one job faster than the other. Once you have partial automation, you still you still usually need a human in the job and you still need to be able to have this level of oversight and a level of context that only only humans can. But once we achieve technology that can do the work for you, I think I think I think it will happen everywhere at once. And I think that's kind of like in in a way slightly
22:57better for the society because we are not in this in this weird world where oh like one job got automated the other one didn't and how do we how do we really like navigate that um all the jobs will be automated at the same time basically that's kind of the conclusion not automated but like somewhere yeah the the physical aspect that that's one one kind of I having the big divide is like jobs that can be done at the computer and jobs where you actually need your your muscles and move stuff
23:26around. That's probably one big divine I am I am imagining and everyone who works with hardware knows like robots will take a while longer for it but at the same time where there is so much investment so much work done in the space that also will happen eventually when I guess like when you're thinking about the business side of things like do you find so nowadays like I feel like whenever I chat with a founder typically people's opinion are like um you have to
23:55chat with the customer first like hearing about like what the customer are like and who are your customers like are they like I guess like enterprise or are they like other um researchers and um how do you think about um the I guess like creating generating like um revenue side of things because obviously you guys are lab and it probably take like a long time compared to um you know it probably takes a long
24:23time to actually um generate revenue.
24:28Um what's your kind of like thought process on like building the company?
24:30What are the momentum that you're kind of like chasing to like maybe raise the next round or like do stuff?
24:36First and foremost, we are a research company. So largely working on delivering the breakthrough technology even before thinking about commercialization. I don't want to participate in the existing agents market of training transformers and deploying them. It's not it's not what we what we set out to do. So first we need to build what we want and then we can we can deploy it to the world. But I am largely thinking about enterprise customers and the companies that are
25:05interested and willing to undergo the internal transformation of moving the business towards more and more like automated way of doing things. And again it will be gradual. It will be step by step. But businesses already are playing with using agents in various role functions and I would like to offer them through our different technology something that is just just much better for their use cases than than the than
25:32the existing setup and like my ideal way of charging it would be per rack. I think I think a lot of a lot of businesses they they like having like hardware on prem and like you know I would like to say this is like uh GB300 and VL70 to rack from Nvidia that you can put in your in your data center we put their core automation software and that software will learn from your data and can perform all the business functions that you need in your company.
26:01this is this is what what we would like to do and like I don't think charging charging per token probably makes sense in some cases but also like very often like users are usage conscious and charging per token suddenly like makes you feel share please we're streaming sorry oh my god my bad I didn't know this like I mean we only came uh we I typically only rent this place for dinner so typically like
26:29these kind of things don't bothers us but today it's been everything has been happening so uh which makes the show very diverse diverse we only got interrupted sometimes [laughter] [clears throat] sorry okay so that's that's okay yeah the per rock the rock it was like per rock [laughter] yeah per rock building model where like you basically buy your own hardware with your own capacity is only serving for you and because the model is customized
26:57like that that rack has a memory and that memory integrates the data from from your business and is used for for for your usage and you know that your data doesn't leave your systems. It is not something that that that that goes to some other company that you never heard of and it's hard to trust. So I think it's it's a good business model of the future in the end compute drives it and we want to be kind of like extra layer on top of Nvidia chips that that help you achieve your business goals.
27:26Let's talk about Nvidia. Um you guys have a great relationship with Nvidia apparently. So it some people are both investor and client at [laughter] the same time. So how how do you kind of like think about um [clears throat] the Nvidia layer? So obviously there are like a few people on our podcast also like are building Nvidia Slayer. Um I obviously Nvidia is still like one of the like not well like the biggest um
27:55provider in the market. So like um what's your thought process on like these other kind of like um players in the industry their approach to GPUs and um how do you think about like maybe expanding the solution into different things as well?
28:11Yeah. Yeah. That's that's a great question like first and foremost at least today Nvidia builds just just the best training chips in the market and that's why we are we are very happy to be working with them.
28:23Um there are many companies trying to compete in a hardware market and it's hard for me to say much about like how that will play out. I'm just not an expert in a hardware market. I'm always very happy to be using the best hardware of the day and make sure our models are will be running very well in there. Um there's a there is a reason why we didn't choose to build our own hardware.
28:45we are willing to devise and develop all of the deep learning stack and all of the like ways of training models but we like the the hardware layer is where we where we are starting. So we we take the existing chips and our goal is try to figure out how to develop the best model with the maximum amount of like learning capabilities on the existing hardware which like largely is built to uh execute matrix multiplications very very
29:11fast. M um thinking about I guess like your day-to-day as a CEO, um what's the difference between like being the researcher yourself to like running a company? Because I I would assume like you seems like you would enjoy research.
29:30I do enjoy research a lot. It's one of my favorite things to do. Obviously being a CEO and building your own company does require a lot of a lot of other things like you know one thing is like you need to secure funding for your company. you need to like be some of some external facing and you need to be able to tell your story and explain to all of other people then AI companies of the day that don't really exist in a vacuum and how do you fit into the ecosystem and how you relate to other
29:59companies is also very important and there are also a lot of operational things that you that you need to do in some way when you are a CEO of a company everything is your problem and everything is your responsibility not just research research is [laughter] the only one of the things I care about the most. But I have a lot of a lot of other headaches and a lot of other things I need to worry about every day that are just just just beyond research. But in some way it is an extremely enabling thing to do to try to design something
30:29that you want to be built which is this new type of model and then try to bring it to the world. The reality we live in is a starting a new company is one of the best ways to achieve certain technological goals and then companies are are not just models not just training but everything that needs to come together it's an organism that's that that needs to needs to function very well and that's what we are that's that's what we are doing. um
30:58what motivates you since like you know you want to like I don't know like I don't think I mean I listened to a bunch of your podcasts but like I didn't know if you ever shared the story of like coming from Poland and is there you know how you kind of like grew up and then like how you kind of like do you always know that you want to become an entrepreneur or obviously it seems like you know you have a you know great like trading career and to like you know riding the wave of AI but what kind of like you know is something that you enjoy are doing and then what do you
31:27identify as your like unfair advantage as a as a founder?
31:30Yeah. Yeah. Like I think like what what what mostly drives me in many ways is I just like building technology. This is to me what makes me get out of bed and what makes me want to do things. I think building specifically new world problems [laughter] not AI but I very much very much enjoy uh building things and solving problems using science and engineering. And that's that's kind of what I just just like doing and what I want to be spending my days. And the the rest of
31:59the questions how in which configuration for the rest of the world I want to be doing that like I have thought about being an entpreneur on and off throughout my life. It's usually a question. Is it the right time? Do I have the right idea? Is the world ready for what I'm trying to do?
32:17The word is not ready but [laughter] it's it's it will be ready soon.
32:21Exactly. Exactly. There was I didn't really have an idea for a company starting in college which I don't think it was a very good company because it was mostly meant to be a company making like personalized and interactive like study materials for mathematics that was my first interactive mathematical that sounds like chip exactly [laughter] in some way working on AI and chd helped me realize that and I think that is that is pretty nice I I think I think back in the day there
32:50wasn't really a good time and there wasn't really a very good very good moment for this idea. These days probably like a few hours of vibe coding and you already can have a prototype of of exactly you were just like tenure earlier.
33:02Exactly. Timing timing is very hard and one of the most important things in starting a company. The question is it is this the right time to build the company I want to build is extremely extremely critical and you don't want to be too early and you don't want to be too late like are you in the right time?
33:19I I assume so. Yeah, that's that's why that's why I decided to do it. I was I was thinking that the window is actually very short for starting a type of company I want to do. So I I I didn't have much time to hesitate.
33:30The first and foremost uh reason is you need agents to be good enough to kickstart you without agents. Starting an AI lab is a very slow process and requires a lot of time. If you see the neolabs that started year ago, two years ago, they were moving in a glacial pace initially. the amount of systems you need to do and the amount of work that you need to build out to get anywhere is just just just a lot. With like modern AI solutions, we're able to probably
33:58compress two years of work in 3 months.
34:01It's it's amazing how much how much quicker you can you can move right now than than ever before. And the second thing is you don't want to be too late from a perspective of already someone being pretty far advanced into some types of research that we are that we are doing. I think we are leading the guard in to finding something that is very different to transformer better than transformer in agentic workloads.
34:24But eventually other companies will be trying to ask the same questions. They are they are I think I think right now today still busy with with with training transformers but that will that will definitely change.
34:36Um how do you meet Rohan?
34:39Uh I plug from our show. like actually Roan let's look at our subject my co-ounder great great great scientist and great passionate of of deep learning I when I was trying to assemble a crew of for that company you can you can imagine you give people phone call that you roughly know and roughly know their research hey I'm assembling a crew this is what we are trying to do are you in or not and Roan was one of the people who answered that call and said like you
35:08know you Jerry I'm in oh wow Oh, like no one. Oh, I mean like I'm sure like is there anyone said no?
35:14No, there are definitely some people that said no. And like you know our our kind of like ratio of of of people who decide to join like is fluctuating over time but but yeah especially early on we had a pretty good close rate.
35:26What would you think is like what makes people join you guys? Is that just because you guys are solving the word class the word first like the first word problem right there?
35:36Yeah. Yeah. I at least one thing we have a very good time with hiring excellent researchers because I think we are one of the companies with both an ambitious research plan solving new interesting problems and actually we are able to pretty well articulate it for those who uh come long enough in our in our journey and definitely for other roles we are we are like slightly more like eye to eye with other companies if you
36:06work on on on systems or on data at core automation. The main part of the story is you will try to develop those things in as aic way as possible using as much automation. You will be trying to become a person of that does the equivalent of a work of a team in other in other places and that's that's our that's our that's our advantage.
36:28Um according to Twitter I don't think you like evil these days.
36:32Yes. I I think eval mostly lead you to good hearting and to thinking about things too narrowly. the word eval is kind of like advertising right so it's like you just pick whatever you're good at then pick the right yeah evaluation things too there there's very little signal in those things everyone who ever ever releases a model they say oh we are good on those evas and they cherry pick and they kind of like present their models as good and the actual like market value of those models is very different to to what those what those benchmark says and
37:00because those are public like everyone just just tries to look good and there are there are much deeper like differences between between models and unfortunately because there's just no really good way to take people are trying all those different models and trying to figure out at least those that want to spend energy on it and people who don't want to spend energy just take the most popular model and run away with it but I don't think I don't think it's even like a very good like way of thinking about about models like about
37:28about evals it's it's a bit of a [snorts] anacronis of times when the models were frozen and were not really like meaningfully learning after after they are deployed So like you know if you take a GPT of the shelf it has some evals because that's that's what it was been trained on but if the model can learn then then those eval scores are different tomorrow even even can be different from minute to minute.
37:50How do you use I mean how do you know if your stuff works well?
37:57Well I know I I test it. I try it.
38:00You try it ideally ideally on the on the work that we are that we are doing. if the model does for me what I wanted to do then then I am a happy person. If it doesn't that's very model.
38:12I'm just [laughter] kidding.
38:14Um what is um I guess like when you're thinking about building a successful lab and building a successful company. Let let's talk about the researchers right who are the good researchers? I mean like obviously like I've done some like researcher events myself and uh I think like in the market or like in the you know in the startup land people are trying to hire people from like open eye or like anthropic um but even like everyone's title is like member of technical stuff and now nowadays there's
38:44the member of nontechnical staff so like how do you like who is the right person?
38:50I mean obviously people publish research papers and how do you identify who is the right fit and then you mentioned about you ensembling your own squad like people have different functions and what are the functions are like typically the most difficult to hire versus like the most easier to kind of like fulfill.
39:07I I don't think there are that much.
39:09It's mostly like um an intersection of a specific timing and who is in your network of the roles that is easier to hire or or or who is more more enthusiastic. But there are in general there are multiple roles you need to fill in the lab and there are obviously in the world people doing that. The question is is is there right person with the right skill set particularly right now interested in what you are what you are doing in your in your company. I I think what makes good researchers is one thing like they understand what they are what they are
39:38doing what they are working on this is like you know probably the biggest difference between like let's say junior researchers and senior researchers like the time you spend thinking deeply about the topics you work with you you you have somewhat a better and better grasp over time and mental understanding of neural networks and their dynamics of training you generally want to hire people who avoid group think and who avoid just following the trends.
40:06It is it is difficult but that's what you want specifically for researchers the most like the the researcher that just thinks what the what what everyone else is thinking. It's not not really a very useful researcher because research is about like solving things that are unsolved and try and go beyond the current methods which which that is kind of like tricky in the current world where the stakes are very high and the dynamics between between companies is this is what it is.
40:31but is necessary to succeed in my opinion. And last but not least, the most successful researchers generally understand systems and understand GPUs pretty well because that's in the end is the substrate we are working with and that's just the layer where from which all of deep learning comes from and we need to be able to combine the insights we have about about the neurons that we are training with the math models that we are executing
41:00and how do we how do we find the good good middle ground between How like I guess like when you're evaluating the researchers like what does a researcher dayto-day kind of look like and then how do you set up like I wouldn't say benchmark but like milestone for everybody's research since I feel like researchers like it sounds like it's kind of like people are um creating the solution for things and um doing a bunch of tests right um how long would it be too long for someone to like
41:27design something new and would it be like would you judge based off like current like certain like feature or like how do you like you know benchmark not benchmark but like how do you like know like people are actually performing on things?
41:40Yeah. Yeah. Like in in some way the most important thing about researcher performing in a in a lab or in a or in a company like ours is just like do they get stuck on things or not? This this is like what probably the number one failure case of researchers. They just look at a problem. They don't really know how to solve it and they keep like doing things that don't move them forward. If they are stuck in a place
42:09too long without gaining any new knowledge, any new insights like this this just is not going anywhere. And I've seen like teams, groups of people just stuck in place and kind of like being on a treadmill trying to run and not being able to to move forward. A good researcher like tries to understand pretty quickly. Do I have a way how to at least like advance my understanding of something? Are there questions that I can know? And even if the problem is
42:38very hard and very kind of like feels distant, usually you need to have like some way how to get to the next step and how to how to get to the step afterwards where you're you're sure you are making you are making progress. That's that's kind of like something that that I think is very important for researchers and to me like the question is do we does the person doing their work increase their understanding of what they are working with periodically and that's already that's already pretty great and like you know the the other ones is just just
43:07like not thinking too small like you know there there's some like discretion of like which pl which direction do we go today which direction do we go do we go tomorrow are they are are they good in interpreting data and are they are they good at being like in some way skeptical enough of not being fooled.
43:26Randomness has fooled many people as well. It comes from comes from experience and it's a skill that you need to that you need to get to and usually if people combine those things they they they end up being pretty successful. The machine learning is extremely fruitful place right now. If you are if you are smart, if you have a good few good colleagues around you and a number of GPUs, you almost surely can um discover something that no one has discovered before that is being fully valuable.
43:55Can we talk about like the talent landscape? So are the p best researchers are at these like really big labs or is there something that like people are kind of overlook or do people do you guys straight up hiring people from like all the top schools and like how do you think about like the experience versus like um I guess academic experience as well like a tech academic experience as well as like the work experience and versus like someone just like have absolutely zero experience but they're
44:24like you know really crack coder.
44:26Yeah. Yeah. I I don't think anyone has a monopoly on talent. There there's an interesting don't have a monopoly.
44:33I don't think I don't think we have monopoly on talent. I don't think anyone has. I think I think the talent has a natural ability to disperse. I I think like you know Frontier Labs definitely at some moment where where where we're the absolute hot bed of of of the most amazing people. Sometimes you don't even know if this is like those people are amazing or just good place at the good time. But definitely people being there have built like absolutely amazing things and had and had great achievements. But many people like you know will agree with me when they say that throughout growth and throughout
45:02years the the the talent density of of at least like very big companies have slightly slightly lowered over time. But the most like I think the more damaging thing is that they've become slightly more bureaucratic in various ways which just just lowers the agency when when a company is smaller it is easier for an individual to achieve like tremendous outcome. The bigger the company is the more the more the teams or structure there is. it's kind of kind of harder to
45:31be able to to achieve that and [clears throat] that's that's why like talent disperses a little bit and there are many many people like starting startups and joining startups and trying trying to do things like that which is I don't think like there are one or two startups that have the best people I think I think it's kind of the the opposite like almost every startup has a few really really great people that that have achieved things before and are clearly excellent and they're building their own teams and their own groups of people trying to bet on the on the new
46:01generation of researchers that will drive what this startup will will will do. So I I think like you know in some way like I think I think on on base levels like the the talent is somewhat like equally spread in the system with obviously like you know various like there there are some companies that maybe like know slightly less talent dense some companies slightly more talent dense but but extremely extremely talented people seek for the places where they can achieve like you know the
46:29most with their time on earth and with their with the brain power that they have and that's that's kind of the optimization function for many.
46:37Is there like different milestone that you kind of set your mind to for your lab? Cuz like I think the thing is like um Anthropic and Open are like the first kind of like wave of labs that like we can see that there's a potential for them to like IPO and obviously it takes them like a lot shorter than like a lot of other companies. I wouldn't say shorter since I feel like tried like open I started like you know seven eight years ago and
47:06um nowadays like you know we have like companies like stripe that takes like forever to like IPO and obviously like it's just like a really different range of like um the timelines for people to achieve that status and nowadays like I believe it was like 300 AR R 300 million ARR or something like that to um actually achieve that status like I'm sure like in the next 10 years it will like shift um in your mind like is there any like different kind of like target time zone
47:34that you kind of like think oh like we're going to spend three years on like X like which is like developing like I don't know like the best like model and like lab and um refining the technology and like maybe like the next like three years on like GM like whether it's like creating like a consumer version of the product or um stuff like that. So like do you have like a general like really concrete plan or do you feel like it just like it's more towards like you implementing the dayto-day now and then
48:03you will see what's happening. I'm I'm sure there's also company probably starting trying to buy you guys at some point [laughter] slash like maybe yesterday I don't know but yeah yeah yeah no no I I think like you know we we have internal road map pretty precise but at the same time it's not something we really talk about publicly what we are what we are doing right now and what we want to achieve by the by the end of this year but like you know those things those things we have we we we have set and we we know where we are going in terms of like building
48:32companies I think Antropic has achieved this this like incredible incredible achievement of building a trillion dollar company in 5 years and many are saying like right now for looking for their for their investor materials that they are probably the most efficient scaling company in the history of capitalism which is which is something and definitely definitely an achievement but I think we live in a in an exponential curve of acceleration. So I do believe there will be a next company that will be even more efficient and
49:01that will will scale even faster. I believe that there will be a next company probably already has been started the next company that will become a trillion dollar company in like three years. Do you know who these people are?
49:12Which I mean like the next generation of these companies some core automation. [laughter] We have some but let's see how the future will go. But I I I think at least at least one of those companies will um um so I want to be mindful of the time maybe we do like one real question.
49:29So one real question would be um given today's like how much do you know about AI like what is like the most like undervalued technology that you you think in the AI arena
49:46generally the network architectures and researching that I think it's it's it's severely undervalued people forgot about it for a while and and it will have a comeback um it will have a comeback back like crypto. You don't believe in crypto? I'm just kidding. [laughter and gasps] Um, okay. Fire round for you. What's your favorite book?
50:11No way. What? [laughter] Okay.
50:14You just said Fire Round. So, I said the first thing that came to my mind.
50:17Uh, what's your I can't believe you read the book.
50:20I read the book. Obviously, I read a book.
50:22Um, who would you invite to a dinner party?
50:28Um, who made the biggest impact in your career?
50:34Good question. Probably some old one.
50:37Where can we find you outside of work?
50:42Uh, at home. I don't do many things. I just I just work and spend time with my family.
50:48Love that. Well, Jerry, thank you so much for coming on the show today. It was such a fun conversation.
50:52Thank you. Happy to be here. Let me I think Aar is going to end the conversation for us because wait I end at the exact
