0:02It's with great pleasure uh that I introduce Andrew Ing uh as our second keynote today.
0:13And for me really this is a welcome back. So welcome back Andrew. He gave uh the second key second ever keynote at Bayern back in 2012. Um and I knew Andrew and interacted with him quite a bit when he was at BU and BU. He was when he was at BU he was organized and hosted Boom there as well and he had some amazing keynote speakers with Jeff Dean Yan Lun and Trevor Daryl at that
0:41time and so welcome back after 14 years. A lot has happened in this time. Uh it's great to have you back.
0:53So Andrew doesn't need a long introduction but I want to give him some sort of more introduction uh and then I'll hand it off to his really interesting talk. So Andrew is a pioneer in machine learning. uh he was a founding lead of Google brain team and he served as VP and chief scientist at BYU where he led a 1300 person AI team responsible for BU's AI technology and
1:24Uh he was also the formerly the director of the Stanford AI lab and was named as one of the most influential AI persons in the world by Time magazine in 2023. I feel that's still the case in 2026. And I'll go through his titles although he asked me not to. Uh Andrewing is the founder of deep learning AI. He's the managing general partner at AI fund. He's the managing partner at AI Aspire.
1:53He's the executive chairman at landing AI. Uh he's the chairman and co-founder of Corsera. And he's also an adjunct professor at Stanford.
2:05So in addition to Andrew's work on machine learning, he's also a pioneer on online education and we're just chatting about this uh right now. We feel that education is still an important part of being a human. Uh Andrew has served over 8 million students, have taken his classes online, and he's changed countless lives through his work at Corsera and his work in AI in general.
2:31So, thank you very much for all that you've done and all that you're going to do. Um, I'm going to hand it off to Andrew. He's been quite kind. He's going to have a a bit of a shorter talk today to ensure that all of you have time for questions. So, thank you, Andrew.
2:46Okay. Awesome. Thank Thank you. And it's really good to be back. um looking back on the abstract of my last Baylor and talk um was uh talking about deep learning and the early days of what running Google brain at the time was like and it is uh really nice to be welcome back. What I want to do today is talk about something slightly different.
3:07I know that many of you in this room are students and rather than talking about my research, I actually want to talk about your research and in particular over the last uh few months my team at deep learning.ai has been studying extensively how AI engineing skills are evolving uh because of AI. What we can build and how we build is changed dramatically. And so what I want to do is present to you what we've learned um
3:36of AI engineing skills and then also share with you some analyses that we're running this past week in preparation for this meeting um for how that shifts when it comes to the skills that AI researchers are being asked to demonstrate.
3:51of all the different fields that AI is affecting, I think software engineering or AI engineing or software development is the one that is most heavily affected mainly because of AI coding agents. Um, and so I think of what we see in software engineering as early science as a harbinger of what we'll see in many other disciplines, including maybe how to do AI research, but also more distant fields like how you do marketing or operations or HR or sales or other types
4:19of um, especially knowledge work. And so um I have relatively few slides but what I want to do is first share what I see happening in software engineering um and then we'll map that to things I'm seeing with a mix of data and maybe opinion on what's happening in research. The state of software engineering it feels like we have many wonderful building blocks by which I mean software components modules APIs we're going to assemble to build
4:47wonderful software very quickly. And when I think about building blocks, if you have, you know, one type of building block, you can build some stuff, but not as interesting as if you have a couple different colors of building blocks. And if you mix in some more blue building blocks and then some red and yellow ones, then what you're able to build grows exponentially or grows combinatorally as a function of the number of building blocks you have. And
5:15with a handful of building blocks, the number of ways to use say coding agents to combine them to build amazing things um becomes very large. And so that's why I think this is um such an exciting time to be building software. And by building blocks, I mean both AI building blocks like LMC can prompt, agentic workflows, build harnesses, um rag, uh eval analysis, which I'll say more about and so on. um as well as nonAI building blocks like UI components, databases,
5:45identity of off, maybe payment, observability, and so on. Um and with all of these wonderful building blocks, my team start to see what was it like a over a year ago, like a year and a half ago, um a very funny phenomenon start to rise in how we were building and we wound up calling this the product management bottleneck. But the way that a lot of software is built in uh companies is we'll have a product
6:14manager decide what to build. So make decisions like do we want to target current college students or graduates or working professionals or parents or kids and then what are the features they want? Do you want a button here? Do you want a text message notification versus inapp notification and so where product managers make all those decisions and then you know an engineer would implement it. That's been the traditional workflow the division of labor between PMs, product managers and software engineers. And what we started to see like a year and a half, two years
6:42ago was this um was that uh we often have this process of building software where um you know we'll write software, we have an idea, we'll implement it and then we will take it to users to get feedback. Uh so we think of that as product management work and based on those learnings we'll go back and change our conception of what is the most valuable software to build and we'll go around this loop and iterate a bunch of times in order to hopefully create
7:09software that someone loves and um with AI coding this became really fast and increasingly um the user feedback became the new bottleneck. And so I remember like a year and a half ago, year and a half, two years ago, I started to see this very strange phenomenon where um the designers and product managers on my team were being really stressed out because they would design something,
7:38hand it to an engineer. And previously an engineer would take a week to implement it and then get back to them to say now what? But increasingly engineers were getting done in half a week and then two days and then a day and engineers are constantly saying all right, I'm done. Now what do you want me to do? And so a lot of product managers and designers were you know uh great designers, great product managers but because engineering had accelerated so much it was difficult to keep up with writing specs for the engineers to to
8:07you know keep keep them busy. Um once upon a time there were engineers whose row it was uh to take a pixel perfect design you know maybe designed in Figma or in some drawing some pixel perfect design of a app and the role of the engineer was to just build it according to that design. I think we're now well past that era and today I will frankly not hire any engineer whose role is just
8:35to implement to a spec that they handed and instead um starting a couple years ago but increasingly now engineers should no longer just be building to a spec but instead participate in making product decisions and design decisions deciding what to build and how it should look and feel user um and in shaping the build and in kind of steering ing the overall build process. One of the themes of um that I see in a lot of different
9:03job roles is that AI automates a subset of our work like the coding part of it um and and especially the verifiable parts of our work which I'll touch on later but this creates more value for the complement of that and um and so the fact that coding the fact that building is so easy is creating more demand for deciding what to build. I also map later to how this is shifting the way we carry out um academic research as well. And so
9:31um with shifts like this um I find that in software engineering the main uncertainty is what users want right so if writing traditional software so not AI software you write code and for the most part the code does whatever is told you know I don't mean to say code is completely deterministic you have network outages databases can run out of space the code isn't completely deterministic but for the most part if you write a piece of software you in
9:59Python, C++, if it's not a machine learning piece of software, it mostly does what it's told and we can kind of have a good mental model for what's supposed to happen. Again, not dismissing. We have weird concussency bugs, security issues, but for the most part, we know we can more predict what software is going to do. And so, a lot of the processes for how we build software are designed to address our uncertainty, the fact that we don't quite know what users want. And so if you study things like um agile software development methodology or how to run
10:29sprints, a lot of it is got to build stuff, get user feedback, you know, as part of the process of figure out what is something that users will love. With AI software, um there's one other major source of uncertainty. And to me, this is the biggest difference between AI and non-AI software, which is um AI systems are made out of code plus data.
10:50you train a supervised learning algorithm on a bunch of data or you train an OM or someone else has trained an OM on a bunch of data, pre-training, post- training, fine-tuning, uh reinforcement learning, whatever. Um, and because none of us fully know really what's in the data and how a code will interact with the data, this creates another very important dimension of uncertainty for how AI software will behave, which is that you never quite know what a supervised learning algorithm will do when you train it. you
11:19never quite know what predictions it'll make on new examples or even maybe not even what it'll do on the training set until you run it. Um, and certainly when you prompt an LM that's been trained on a lot of data and fine-tune and post-train on some, you know, complex process maybe with regenerate his own data through RL um you don't quite know what it will do. And so this uncertainty in what AI system will do is to me the key defining characteristic between what
11:47it's like to build software systems and nonAI versus build AI systems. And so a lot of the processes for how we approach building AI systems is to come up with statistical ways to measure and govern and then to also to steer the performance of these you know sometimes unreliable AI components to make them reliable. Um, and so, um, and that's what's makes AI development so iterative, right? You never know what you're going to get. So, you have to build it, see what you get, and then use
12:17that information to keep on steering it. And so, kind of motivated by these observations. Um, my team, Dunder AI, we analyzed over 20,000 job descriptions. Um, did dozens of structure interviews with senior engineers that we respect.
12:34uh also ran a bunch of her own surveys to try to synthesize what are the most valuable AI engineing skills and I want to use the term AI engineing skills because these are rather rather than the AI engineing job because um AI engineing skills will be important for people of many different job roles not just people whose title is AI engineer. So software developers, front end, back end, DevOps, security engineers, all of these people
13:03um will be more productive and get more done and I think have more fun as well with AI engineing skills. And then of course there will be a subset of people with an actual engineing AI engineing job title that will especially benefit from having these skills. But um in in our research kind of you know maybe informally think of it as uh getting a massive amounts of data and then running clustering to figure out where are the most dominant clusters for from from from very messy data. So human steering
13:31as well in the process. Um we I I I felt that the four most important AI skills are building and deploying AI applications. Um and again the key characteristic of building and deploying your applications is knowing how to manage these uncertain systems with slightly unreliable slightly random output and to wrap around it the systems processes eval maybe determiny code to take these slightly flaky components to
13:59turn it into relatively reliable software at least reliable enough for your application. Um and then uh software engineing fundamentals. It turns out that um I know there's a lot of hype you know that we get coding agents write all the code maybe you don't need to know code designs anymore or something. I think that's not true.
14:18What I see is that um when people vibe code without understanding often um you don't they they don't even know what tradeoffs there are to be made and a coding agent is implicitly making some trade-off in terms of security, reliability, uptime, cost, maintainability, code quality. Um and unless you understand maybe one layer underneath the hood how things work then um you don't know what trade-offs exist
14:47to be made and some you know vibe coding AI thing will make a tradeoff on some point that may or may not be optimal. Um and so this is why all of the best engineers I know all the best software engines I know have deep software engineering fundamentals. Uh and then the flip side of this is um another common theme that first rose up in software engineering but is now increasingly in other job roles as well
15:13is um because it is now so much easier to build software than before. I see a lot fewer front-end developers or backend developers or mobile developers.
15:24Instead, most people have become many people are becoming full stack developers, meaning you built the entire end-to-end process of a of a of a piece of software, right? And so, um, I find that in a lot of job roles, we see this outside engineing as well. Um, a lot of people are able to rise up and do a bigger scope of work that used to have a require a larger team to do. Um, and I'm going to go into this in a little bit more detail, all four of these in a little bit more detail. um using coding
15:52agents. It turns out that uh this does not show up in job descriptions. I think the the you know coding agents of of these four key top level skills, coding agents is the one that's evolving the fastest. Um and so I find I feel like the way we write code really changes quite significantly every three months, right? Definitely every six months, maybe more than every three months uh but not every week. I think that hype is also, you know, just just not true. Um but it's been interesting how uh in you know the way my teams operate, the way
16:20we interview senior recruiters or senior hiring managers, everyone kind of agrees we have to use coding agents effectively. But this is actually one sector that is uh not yet showing up in job descriptions. Um and then also increasingly we see job descriptions also engineers that um don't just implement what you're told but can participate in shaping the build. So because I think the engineing skills map is a important catalog of the things
16:49that maybe all of us um if we're skilled that will let us get more done maybe be qualified by more jobs um I think also really you know have more fun because frankly I don't know getting a job is nice but having really fun work to do right that that also seems uh uh uh as important. I want to just very briefly run over these things in a little bit more detail. Right? So building applications um underneath that we mapped it to uh six key skills. Uh Elm
17:19foundations um grounding models with data uh uh let's see building agentic systems um evaluationdriven development operating production and machine learning foundations. It turns out that when we look at our data on job descriptions, um there are two to three of these that are elevated in importance for those of you that want to be a researcher. have another on this later but just um it turns out that while these skills are important for you know
17:48to for you to become for you to become skilled at AI engineering the two or three of these that are elevated importance according to our data of what employers are looking for for AI researchers is machine learning foundations especially uh followed by foundations um and then slightly weaker but I think is also true is evaluationdriven development and I think this probably kind of makes sense uh machine learning researchers or AI researchers need that deeper understanding of the foundations of
18:16evals. Um and uh let's see. I want to share one one more thing about this. Um I find that one of the most difficult aspects of doing this well is that the skills needed are very stage dependent.
18:31Meaning that if you are building a piece of software and you're a very early stage project, you know, say you're a student and you want to try something out. um the way you approach that is very different than if you're engineer in a large company about to ship something to a million users. And so what I find is that um the best practices for a zero to one project meaning you're starting from nothing or building something for the first time kind of quick and dirty protoite the processes and mindset for that is very
19:00different than if you're building a you know robust enterprise grade piece of software that really should not fail and has huge consequences to to many users or users revenue impact or whatever if it fails. And so to give one concrete example, um it turns out that if you're building an agentic workflow, I see this a lot of teams. I will give you one pain point I see a lot of teams struggle with. If you're building an agentic workflow, say you want to build a customer service chatbot or something to process your financial invoices or whatever. Um very often you could build
19:30a prototype in one or two days or like over a weekend and then you can look at it, see what goes wrong, tweak it, change the prom, add a rag module, remove this model, tweak the harness.
19:41And by kind of hacking at it quickly, you can make something that goes from, you know, make up numbers, let's say 0% to like 80% accuracy in a few days, right? What I and so that process a very fastm moving kind of trying things out uh uh not releasing things in an irresponsible way. So do be responsible.
20:00You can kind of get very rapid progress in days or maybe maybe days. Um what I see is that there's some teams they're familiar only with this quick hacking around kind of process and what happens is it goes from zero to 80% and then the performance just plateaus. It bounces around. You improve worsen improve worsen and you just struggle to get it to do better. And what happens is as a project matures on many of my projects what we had to do was gradually shift to
20:29a different mode of operation where instead of just looking at it as the evals we start to collect a golden data set or write more rigorous evals or write a rubric for how we do evals and then gradually build up those rigorous evals um and think harder about what are the changes is no longer fitting this prompt is instead going deep into hyperparameter tuning for the rag system or building a gent intake retrieval. And it's by doing these things with regular E valves that you then gradually go from
20:5780% to grind out 81 82 83 and then over you know like months you eventually get to you know 90 95 98 whatever and then you have something really reliable. And so I find one of the most difficult things about being skilled at AI engineering is even on a single project the skills needed to move quickly um responsibly but quickly are very different than when you're doing this rigorous eval error analysis driven state error driven to grad just grind out that performance until it hits the
21:27level of reliability. And it turns out um uh you know I've seen people from a big company insist on rigorous evals on day one and that doesn't work well because it's too slow. I've also seen people get to late stage project and are still kind of hacking around without rigorous evals and that also doesn't work because you plateau 80%. But so one thing I found for a lot of these skills is that um the best approach is very stage dependent uh and unless you kind of um uh you know can find a way to gain
21:56experience across the multiple stages of a project is it can be tricky to pick the right move for applying these skills. Um all right software engine fundamentals uh maybe building full application managing data is actually a big one. Um I find that uh we've built a lot of our data infra primarily for human consumption but I find myself rearchitecting tons of my data infra for my agents to use rather than for my personal use. So for actually for
22:24example um I don't know I used to do data analysis you know in Jupyter notebooks you know pandas data frames but now my AI agents do a lot of my data analysis for me but to enable that um is requiring a ton of new data plumbing to make my data platforms agent ready rather than only human ready uh since architecture oh I want to just highlight one other thing making systems secure is really shifting um in computer security
22:52there's movement called shift left where the process security has historically we design the software implement it deploy it and then you kind of secure it in deployment right or you kind of figure out what you want to build deploy it and then secure it maybe later or maybe a different team secure it in computer security there's a movement called shift left which is a move the responsibility for securing software left in this flowchart of work process and what that means is um just like many developers are now full stack developers rather
23:22than silent I find a lot of developers now have to be a little bit of a security engineer as well. Um and so one of my friends Rohit Prasad and I have been working hard on a open source harness called open worker uh with a strong emphasis on supporting security workflows. I think this um folding this into a lot of our day-to-day work is is an important shift and then scaling operating production. Um I'll go over this a bit more quickly. Uh using coding agents, you know, how to direct the
23:50workflow, enable calibrate to the right level of autonomy and give agents more autonomy when you can do so safely. Um how to verify and review the work, customize agent environment, coding agent and and lastly coding agent foundations. Um of these different skills, using coding agents is evolving the most. So, I plan to keep this updated, but I I suspect this is one that I'll probably end up changing the most quickly. And then lastly, uh shaping the build uh driving the build
24:19low where driving the build loop where hopefully your unique expertise helps you decide or shape what the project will do next. Start to make some product decisions where um uh shaping user empathy so you can build something, go to users, get some feedback. It doesn't mean engineers should become product managers, but a being able to make some of those decisions um accelerates projects tremendously. Um and then communicating and leading and I I I feel like some of these things have always
24:47been kind of on software engineing job descriptions, but some of these things seem like they are elevated in importance relative to what it was a few years ago. uh and then lastly high agency ownership where um I find that because building is so fast in comparison communication cost has become higher and so the value of having people that are very high agency and own things that would drive things forward in responsible way has become really valuable. Oh, one fun thing. Uh, some of
25:17you will know that like a what early 2000 like a two and a half years ago, three years ago, I'm I'm the one that had coined the term agentic AI. Um, actually ran a campaign in the batch to try to get people to adopt the word agentic. And I wasn't expecting, you know, agentic to take off the the the way it did. But one funny thing I saw in a bunch of job descriptions was another application the word agentic where I see number of job descriptions asking if humans that are highly agentic. Um
25:47wasn't expecting to to to see that maybe app description as well. Um and underlying all of this I know there's a lot of hype about RSI and maybe you don't need human research as human anymore. That's just hype. I really not seeing it. But one of the biggest shifts underlying this is um shift from human work to verifiable work to unverifiable or to judgment oriented tasks. And it turns out that um OM are very good at verifiable tasks. And our ways of
26:16training OM for verifiable tasks are also much better than training them for unverifiable tasks. So verifiable task is something that has a objective you know right answer or right answers that can be automatically validated such as is your leak code solution correct that's a verifiable test or is your math problem solution uh correct that's a verifiable test it turns out because we can run reinforcement learning gyms or we have objective reward functions or
26:44objective ways to label these things um um mechanisms for improving own performance on verifiable task test is much higher the ability to improve its performance on unverifiable tasks.
26:57Things like what's the right software architecture or what's the right marketing slogan or what's the right way to approach data collection. Those are unverifiable tasks or judgment tasks that do have better and worse answers but there's no one right mathematically correct way to approach how do I approach data collection for this and so elms can approach unver judgment tasks right we can use supervised fine tuning we can get human experts and a ton of data so it is possible for LMS to help
27:27with uh judgment tasks but because we can't run our gyms on them and because data collection ction is more ambiguous is much harder. Progress is much slower on improving our performance on judgment tasks. And in history with mostly overgeneralized AI's capabilities, uh when IBM beat Gary Kasparov, a world chess champion, Deep Blue, people were overoptimistic about how well generalized from chess to everything
27:54else. Um IBM Watson, Alph Go, society tends to be over optimistic and how well AI would gener generate from solving Jeopardy. IB and Watson to everything else. And at this moment in time, um I think uh performance is clearly amazing, right? It's great at verifiable and unverifiable tasks. But I think we are as a society probably overgeneralizing our optimism from the ability of this jagged intelligence to do verifiable to unverifiable tasks. And this is why many
28:24experts have this common experience that when we ask Elms on something that we don't know much about, it sounds great.
28:29But when you ask on something that we are expert on, yeah, this judgment is kind of questionable. Um, and so but what I see underlying a lot of this is um an educational system as well a lot of our day-to-day work will shift from the verifiable tasks like writing correct code to a lot of the unverifiable tasks. Um, and so the way um, uh, I think about learning these skills is the judgment tasks that underly a lot of these things. Um, and
28:57because humans have a and and it turns out for a lot of judgment task, we make those decisions by pulling in a bewildering vast range of context. So when making a product decision, I might remember that someone had a funny facial expression when I said something to them like six months ago and that funny facial expression makes me realize, oh, they don't want that feature, that influenced the product decision. or you may remember some market report that you read, you know, three weeks ago uh and there was a funny point data point in
29:27that report or in that research paper that causes you to influence that judgment. So a lot of judgment tasks require pulling a bewildering range of context um to make that decision and I think for for for the foreseeable future humans seem to have that context advantage in access and pulling out the relevant pieces of data. So um uh and this is why I think uh you know software engine job postings are out right there is no job apocalypse. We we just can't find enough highly skilled software engines and AI engineers to to do this
29:57work. All right you know what let me skip this of time. Um so I want to show you the the data on uh research jobs versus other AI jobs. So um of these skills if you look at the research job postings that you know kind of we we we found online the ones that more emphasize uh the ones that more heavily emphasizes machine learning foundations significantly more highly emphasized uh followed by foundations and then weaker
30:26maybe a little bit eval driven development um under emphasizes soft engine fundamentals uh which you know uh I think when you're doing AI research Maybe you're building full stack software less often. Although I think in my own work I find that my ability to quickly build full stack applications also helps with the research process depending on what I'm trying to accomplish. Um there was insufficient data on coding agents. So I don't have numbers on that. And then um one thing
30:53that uh one one outlier that I actually don't agree with so so maybe um research job postings have much less emphasis on building driving the build loop but I feel like um this is one place where I'm going to make a prediction different than the data which is maybe I think driving the software engineing build loop is less important but I think the research process is changing and driving the maybe driving the research loop um and learning how to prosecute research with AI tools is becoming elevated in
31:24And then lastly, uh just just one last slide, right? Some some reflections the CSAI research process. Uh we can debate what the right process is, but I think you know maybe Lucy when I'm working on research paper um I'll select a project was very hard review literature, come up with an idea which still seems really hard to me. Um uh implement it, run it, eval data analysis, write present and obviously there are multiple feedback loops. I'm just showing a couple of them here. Um I think the biggest change is
31:54our ability to implement ideas and run them. Uh and then I think um data analysis has become easier. I don't know if you all do this uh but I I I routinely use my coding agents to do data analysis for me. uh and it hallucinates make tons of mistakes needs tons of verification but you know it definitely accelerates the analysis process but coming up with evals for most research project is still right really painful and still takes a long
32:21time. Um and then there two other parts of this that people talk a lot about using that I feel you should use it but the impact seems frankly more marginal to me. One is a literature review. You know, various deep researchers um yes it does help but I feel like a lot of the bottleneck is still the human understanding. So it it helps people should use it but I think um is important feels uh greater than my own ex than than my own experience. Um and then you know Elms are great at writing
32:51and yes you should you can use it to help with your writing but this is another place where um I feel like you know uh I I think for non-native speakers of English can be a huge help but I feel like for my own writing um I don't know I I feel like do help but just not that much are curious um you know every week we write a newsletter called the batch and I write the letter myself right it's it's never been AI generated you know have AI
33:18help me proofread and find a typo here and there and maybe have it help me fact check, but I actually write those things myself with, you know, with these fingers. And I find that maybe I just not figured out how to use AI to generate content, but I find that I'm still having to do a lot of my own thinking in my own writing. Um and what this means is that um I think the shift in research there's a very real shift in research and I find that uh at least for and and I'm excited about all the work
33:46on having AI researchers you know I've read a whole bunch of those papers and my friend Dang at Stanford had really nice work on AI for idea hypothesis generation turns out to be less creative but still needs human screening human judgment. So, I'm very excited about all all of those things, but it seems to me that a lot of the bottleneck for research is still understanding the literature and where to look for problems and they come with good ideas, but our ability to test those ideas is now much higher. And so we can take many
34:14more shots and go. But that process of uh understanding the state of the arts and thinking deeply and synthesizing that bewildering range of context that AI is still not that good at um uh still feels like an important thing for us to do until AI maybe gets vastly better.
34:32And I know that when academic conference there's a lot of chat about um you know conferences uh like this and others being overwhelmed with the number of research paper submissions and actually have two minds about that. I find that researcher productivity is up. You know we can now maybe write a decent research paper in one or two months if we're lucky that would have taken 6 months or or 9 months or a year or something. So I actually feel like collectively um humanity is generating more highquality
35:01research uh but at the same time it is also being buried by an avalanche of you know maybe lower quality AI generated work. Uh so I feel like rethinking so I'm actually very encouraged that I think the volume of good research done is higher because our ability to take more shots on go but um the reviewing process is breaking. So again just one one thing that Einang my PhD and I working contro this stamp agent reviewer and um it's actually been very
35:29interesting. Um I announced this I think in like a tweet or something and I I I haven't talked about it much. I talked about a couple times but uh it's still we still get about uh 1,000 papers submitted per day. Um and so we kind of review far more papers than most conferences. And then um oh in our data we also see very clearly you know the uh two or three days before any AI conference there is a huge spike in
35:58submissions. Um and then the other thing that I have very mixed feelings about uh is um uh we we've not we we don't look at the you know uh respecting author privacies. I don't I've not been going around looking at what papers are submitting. I take author you know confidentially if I'm punishable seriously but uh based on what I've heard not on our data but elsewhere I'm fairly confident that tons of reviewers of conferences are submitting the papers they are getting to the agentic reviewer
36:26even when a conference policy tells them not to. So that one I have very mixed feelings about and and kind of sometimes concerned but because we don't want to poke at it and look at what's actually being submitted I'm not quite sure how to put a stop to that. I'm not quite sure we want to put a stop to that either but but so I think with the flood of research papers oh well actually the reason I built this was because I was chatting with some students that um uh needed more feedback on their research.
36:52So the the reason that um I started to put this together because I saw students that you know would do good work and then have to wait for conference submit a paper to get feedback from three reviewers. It was just a very long iteration cycle and I felt that we could build some sort of AI reviewer. then students, you know, with kind of busy advisers, friends are busy could could just have another way to close the loop on getting feedback earlier to drive this iteration, right? But I think uh we're still in the very early phases of
37:21uh um rethinking what the research workflow should look like. Um so that I think uh I just just one one last thought. I'd love to see see take take questions uh which is um I know some people even at the frontier AI labs they're worried if AI will replace them right um but it seems to me that a lot of the tasks that researchers do um
37:50definitely AI today and in the near future cannot automate um but a lot of the judgment unverifiable tasks while I hope that someday we'll get AI to do much more of these tasks for us. I think um the jobs of good researchers feel safe to me and I wish we had a lot more researchers. So to all of you considering do you want to be a software engineer or researcher? I'm actually confident that the world needs a lot
38:19more people skilled in AI and dream and in research and that if you keep on building these skills the students behind you and others um that there'll be a far more exciting work for you to do than I think any of us would do.
38:45Thank you very much, Andrew. We'll open up to the audience for questions. Got the microphone here. I saw one microphone. We've got one in the back.
39:06Um, maybe speak loudly.
39:13Okay, now I can hear myself. Um I want to ask you an opinion on the following.
39:17So I know many conferences are saying not to use AI to review papers. Some of them are starting to relax that. And something I found speaking to some students and some other faculty is that there is this um kind of culture arising on certain students on using LLMs to review the paper and then change absolutely anything they can do so that the LLM at the end tell them okay I finally understand your paper without much problem. And the problem I find
39:46with that is that sometimes they start tweaking the paper in a way that it seems that the LM is driving themselves on what to do to satisfy the LLM because they are afraid that a reviewer might use an LLM and they want to kind of hack that. Um, and at least personally created problems because I sometimes feel I have to discuss with some collaborator saying, you know, this is mathematically correct. Your LLM is wrong. I can show it. Um, but they're like, "No, but what if the reviewer use
40:15the LLM?" They're going to bring it up and then the AC will say, "Well, probably you guys are wrong, another reviewer." Right? And so there's this kind of like toxicity in terms of satisfy what an LM would say about your paper and that's driving I believe more than one multiple groups I think in the field. So um, have you seen that or what's your opinion on how to how to take that?
40:39Yeah. No, I I think it's unfortunate. I think you know unfortunately the problem you describe is real and um individual students trying to solve what is a systems problem feels like putting the student in a very difficult position because if the paper is going to go through a weird review process then they're almost obliged to optimize for a weird review process which is not the position we want to put a student in. Um I have heard from people that felt like their reviewer may submit the paper to
41:08even a Stanford agent reviewer and so they want to know in advance what it'll say and I think that's also I I have very mixed feelings about that as well.
41:15Um I wish I had a solution. I maybe I feel like this is a problem for us to solve at the how academic reviewing should be done process rather than putting pressure on individual students to solve it. And candidly, I feel like Elms have really bad judgment on a tons of judgment tasks. I mean, again, on tasks that I know nothing about, I think my Elm is brilliant, but a lot of things I think I know a thing or two about, I feel like, no, this is not right. That's awful decision. You can't do that. Um,
41:44and oh yeah, and and I find that the the suggestions it makes to my writing or my research papers, I I think, you know, sometimes it's interesting, sometimes it's right, but a lot of times I've rejected suggestions. But but to me the heart of it is then forming those skills to have the independent judgment to decide you know what to accept and what to reject. So I think forming that skill to have that judgment for for the judgment task rather than the verifiable task. So seems like the foundation for
42:12what we need to do as researchers. Yeah. Sorry D. Thank you.
42:22Can you hear me? Yeah. Hi. Thank you for sharing Andrew. So one one thing you have mentioned right is obviously that the um the you know the you know the uh change for developers to think about you know the design process and the evaluation process. Now more and more tools right now you know we're building tools we're adding like open clawite features like open workers and you know the instinct or the u a new sort of open claw functions in there. Do you see you know different skill sets or different development thinking once you
42:50approaching where you've got more more agents and more tools out there that you're developing into your product?
42:57Yeah. So yeah well right I think right tools are changing all the time. uh I think which some of the things I hear teams debating right um I don't know it's still really difficult to orchestrate multiple agents and then when you have multiple people working on a project how do you orchestrate multiple agents under the guidance of multiple developers and then also you know when you um push some of GitHub uh how do you not only push a code but share some of the context that cause you to drive your swarm of agents to do something so they collaborate swarm of
43:26agents at the right context and then do the right thing so I find that um I I I find that a lot of the teams I work with are in frankly like day-to-day debates and and we kind of change our mind every you know two weeks or every month about how to do this. Uh but yeah this is actually a very exciting time actually.
43:42I find that the single person workflow is complex and evolving but the multiple person workflow is just you know feels very wild west right now. Hi. Uh, hi. Hi. Can you hear me? Uh, here. Yeah. Yeah. I have a question that I'm a PhD student at Santa Clara University and I saw lots your blog and about the skills have me gain lots of follower on Twitter. Thank you so much.
44:15Yeah. I have a question about that. uh right now the web coding make everything easier and we can do lots of things with the agentic AI so for a long-term goal is become a vertical or become a horizontal like a talent is a good goal for like a PhD student or other students uh in general yeah oh sorry so what what has become horizontal become a vertical like
44:43focus on specific domain or like learn everything become become a horizontal talent. Which one is better for the students?
44:52Wait, I'm going to give the answer you don't like. I I think it's both. Um and yeah, and and I think it's because um I find that so when just speaking about how um you know, my my PhD students at Stanford often just got to go really deep in one sector, but then um I feel like you know it's not so easy to build full stack applications. let's just learn to do that because if there's an opportunity to apply that skill then you can do it and maybe build a full stack application
45:21to collect data or get feedback or what it needs to be done. So maybe to make an analogy um you know once upon a time society did not have computers and if you're a I don't know um history PhD student right there's probably a way to do a PhD in history without using computers but it's such a useful skill if you have that horizontal skill it lets you uh uh get a lot more done and so I find that having this broad range of skills gives you lots of opportunities to do things that were not
45:49possible a couple years ago and I find it hard to predict where exactly those opport opportunities are but I so so I think uh yeah so so that technical depth is really valuable but then there all these skills that not that that are hot but kind of acquirable they're also worthwhile um to learn and then as usual I I find some of the most exciting work is you know that crossing discipline boundaries those weird connections that no one thought of and I find elms are not very good at making those connections either and so if you have
46:19that broader discipline it could let you spot unique research ideas okay thank you so much. Thank you.
46:25Hi. Uh my name is Sid. I am the principal ML at Pinterest and um I was specifically asked to get your opinion on like I'm responsible for researchers at Pinterest developing genera applications safely. Uh we're building an automated red teaming application. So asked to get your opinion on um what should be the like I like how you said code plus data um paradigm is different.
46:50some asked to get your opinion on what should be the inputs for the red teaming code and what should be the outputs. What are some best practices for building automated red teaming applications so that engineers can ship their gen applications safely because you can't have humans automat manually red team for weeks if they are developing gen applications in days.
47:13So how do we accelerate the process safely?
47:16Yeah. So um you know red teaming is important part of uh safety and finding problems and let me just let me just speak to uh safety and security at an even higher level. So um elms today have problems. They can generate you know bad outputs, erroneous outputs and then in the case of the widely publicized uh uh hugging face hack by open AI right sometimes agents will hack into other people's computers.
47:43Um I feel like all of these instead of viewing these as you know signs of the sky is falling or rogue agents are taking over or whatever the hype is. I think of these as very hot engineing problems to be solved. And the good news about what happened with the open air hugging face hack is that now that you know the whole world or now that so many people have seen that tons of teams are taking sandboxing much more seriously.
48:09And so I'm sure that a similar incident happening at OpenAI today is way less likely than you know a few months ago.
48:15And then also a lot of my teams on open worker uh we're doing work with um Nvidia open shell you know we're building much better sandboxes because this bad thing happened with hugging face that should not have happened but because it did we now all whole community is working to reduce this particular set of problems. So I find that for a lot of um uh uh engineing problems um the only way to discover these problems is to build it and then deploy it in a responsible way. Uh I
48:44think open had gaps in sandboxing and monitoring the the the artifactory loophole I found would have been hard to sandbox around that but I think monitoring was weak. Um so and then um for a lot of engine things including you know aviation right where uh today uh early days of aviation airplanes crash and it was tragic and you know accidents are actually really really bad but repeatedly learn from those accidents to get rid of the problems one at a time until today aviation is really safe um
49:14and so in a similar way we will keep on building and deploying these systems and keep on finding problems and solve these as injuring problems I things actually I think Jensen and Satia also have uh recently said public things about this they're quite sensible um and then I think uh yeah and then a set of practices on uh safety as well as security are evolving so I spent a lot of time on security we want to review code for vulnerabilities um we want to
49:42uh monitor your cloud infrastructure for configuration issues for your infrastructure security and conf um uh do a lot Um, it turns out one of the hard things about the security incidents is you often get weak signals, right? Maybe your, you know, cloud dashboard says, "Oh, something's funny."
50:01And then this other dashboard says something funny. And human security, uh, engineers really have a lot to do by hand to review these signals to figure out what is just an anomaly uh, that is not a security incident versus what is a security incident. So one thing that we've been doing on open work as well Prasad and I is to try to build AI to look at these different incidences look at these different signals to connect the dots more automatically so we can flag real security incidences faster.
50:30But I think of this as the really hard really exciting engine work that we need to do and we just keep on doing it and then we'll get past this security safety road agents are going to cause all hype which is just not true.
50:42Quick followup please.
50:45Yeah, I think we should. Yeah, thank you.
50:52Uh, hi, this is Omid from Oracle. Uh, given the uh uh presentation that you shared with us on rising importance of AI researchers. I was wondering what do you think about uh the startups on automating science and automating engineering that are recently like raising billions of dollars and their ambitions.
51:15Um I think I think it's exciting. I think attempts to automate science is a great thing to work on. Um, and maybe I'm actually really excited by a lot of the automation of physical sciences where you have a hypothesis, maybe a bunch of robotics, go do a wet lab experiment or simulate the material, whatever. Um, as well as automating computer science and AI research. And then I think, uh, automating engineing also, you know, frankly, coding engines are amazing. If we can automate even more of my work is like fantastic. Go
51:44for it. Uh so and then at the same time um automating everything still seems difficult. Um and that's why it it turns out for a lot of the taskbased analysis of jobs that you know my friends like Eric Brenson and others at Stanford have done um for a lot of jobs outside not just research AI looks like it could automate 30 40% of a lot of people's jobs and that's very exciting but what
52:13that means is we still need people to do that remaining 60% um and in fact that 60% that people can do maybe becomes even more valuable now If someday we reach AGI or RSI or whatever the hype says this week uh and AI could automate 100% you know that would be exciting and that's when we'll have a uh and I I say there's a small subset of jobs where AI can automate close to 100% of the work.
52:37So it's very tiny sliver of jobs are challenge um uh but also uh less than the hype suggests. Um but for research and software engineering um I hope AI moves fast and could ultimate more of my work but um for a long time it feels like humans will have a context advantage. We just know stuff. They don't have plumbing to get AI to do. And there's a bewildering range of judgment
53:04tasks. And uh I'm excited if you know LMS can slowly in this peace me jagged way try to automate more of them, but I'm not but but I I I think it feels over optimistic to me to think that they'll automate all of it uh uh in the in the near term.
53:24All right. With that, hey, this is great. Thank you all very much.
53:30I'm I'm sure we could spend another I'm sure we could spend another hour here asking Andrew questions, but we have to let him go. Um, so thank you again for your talk and uh we'll ask the next speaker to come up uh Richard from Similar. Thank you Andrew. Thank you.