0:07Hello and welcome to another episode of the Pukam podcast. I am your host Yangri and with me today in the Zalina studio is Dennis Becker. Great to be here.
0:18Hi Dennis. Uh our regular listeners will have noticed that this episode is somewhat unusual for us. It is only our third ever episode in English and my first ever episode in English for this podcast. Um, because today we want to talk about how the developer experience is changing with the rise of AI and maybe I I don't know if rise even is the right term anymore. It feels less like a rise because that kind of like has already happened and more like a we're
0:47in the dominance area already [laughter] maybe. We'll see about that. Um, and while we could have this conversation in English or in German, um, it's much more fun uh to have a guest on as well. And because our guest uh prefers to have this conversation in English, we're totally on board with that because uh the alternative would have been to do this in French. And trust me, you don't want to hear my French. I I can't speak to Dennis's French. Uh, but my French is like terrible.
1:12It wouldn't be enough for a podcast.
1:14Yeah. [laughter] And uh, with that being said, I would like to welcome uh, Roma Huette. Welcome to the program. Thank you so much for for having me. Uh uh you know I studied German for many years but I don't think we could carry the whole conversation in in in German. So thanks for for doing it in in English.
1:37That's uh totally fine. We didn't invite you here for your German skills but rather to talk about your actual job and when we did the uh research on your like checked you out on what we could find on the internet. um you have done or you've been in developer experience for quite some time now. You've been at Stripe and at Twitter. Um and maybe just as a small anecdote here, when I was doing developer relations at Shopify, Stripe was one of the like go-to companies that we would look at and see like how can
2:06you actually do developer experience on a like top-notch level. So, uh kind of speaks to your work here.
2:12Thank you so much. Yeah, I mean like I spent uh yeah 5 years at Stripe uh building the developer relations, developer experience function, but also obsessing about the product which was effectively for Stripe, the API design, the documentation, the SDKs, all of the tools that would make developers actually love integrating payments. And when you think about Stripe, um integrating payments had always been a pain for developers. It was always the thing before Stripe that they hated the
2:40most doing because it meant, oh gosh, I'm going to have to leave my favorite NodeJS and Ruby and Rails development all of the sudden to integrate with these very legacy APIs. And all of the sudden with Stripe, we made something very delightful and magical to use. And uh yeah, I spend a lot of my time making that uh uh as as great as possible with a fantastic team. And I think what I found the most exciting about Stripe is how everyone from Patrick and John, the
3:08founders, would care so much about the quality of the API and the abstractions that we would put out there. So that was very much in the DNA of the company to have great DX.
3:20And now you're head of developer experience at OpenAI. Um, and what I want to say, uh, thank you for the invitation to OpenAI's developer event in Berlin. We sadly couldn't record this on site in Berlin, but we now get to do this uh remote. And uh you're in from Paris, right?
3:36Exactly. Way we just uh hosted Berlin was the first of our stops in Europe. It was a great event. We did uh Paris uh this week uh after Berlin and then we're going to London uh on Tuesday as well to kind of wrap this European leg of Dev Day Exchange. And of course like people have have probably followed Dev Day SF about four weeks ago now. It's like kind of like our flagship annual event. And we could stop there, but I think for us
4:02it matters also a ton to not just launch things from San Francisco and have a live stream, but also connect with developer community around the world, right? And I think there are amazing developers in Berlin um and uh same in Paris and London. And we want to make sure we also spend time with them to also listen to their feedback and what they need. Uh it's important to like have that feedback for us. That's very much the way we make the products better and how we build them. Um, an example of that for instance was uh, every time I
4:32traveled to Europe the past year or so, people kept mentioning that they wanted to more voice AI, but the accents and the voices in German or in French were not that great yet. And I agreed. And so that feedback was critical into making like GPT live 1.1, which is now much better with the German accent and French accent for instance. So yeah, we love spending time on the road and hearing from developers who are building great things in Europe.
4:57And what is your response to people who ask you why you still need this face to face time? Like lots of development work is becoming more remote, more asynchronous, um teams are more and more distributed. Why do you care so much about that face to face time in person?
5:12I mean the the sense of belonging and community I think is more important than ever. uh especially after like some of the kind of covid years where people had to uh work um you know uh remotely for a while when you look at the rise of AI and and things like coding agents for instance it's definitely true that now you can do a lot with those tools a lot more that you could possibly accomplish before but at the same time the the sense of like belonging the learning from each other is becoming more
5:42critical right like if you rewind just a few years ago um the techn technology you built upon was not changing that fast. Now like all of the techniques that you might have practiced for the past like 6 months are already changing. In fact, I often tell like engineering teams like if you have I don't know your skills or your agents MD files that have been untouched for 4 months or something like that, they're probably far outdated by now. Like the fontier models are becoming so great
6:10that um you want to learn from each other. you want to like actually connect in person, share the best practices, see what the others are building and uh as I hinted at also like um the needs from different people from different regions are also different. we mentioned languages but when we go to APAC that's even more acute right like in in Japan in Korea um the price points of of what we can offer these models for is also
6:39critical for some other regions and so yeah the the community building is is critical and that's why I think we can do some of that work with our team that we've been growing around the world but there's some limits to that and that's why we created programs like Codex Ambassadors for instance and Codex Ambassadors are amazing to us because they are passionate builders using tools like codeex that are able to run events in regions uh that we cannot reach yet you know like there are cities around
7:08the world where we don't have openi employees yet [snorts] uh it may take us a while to get there even um but luckily we can rely on these codeex ambassadors building events and yes this sense of belonging is really beloved by the community if we're talking about like developer experience it already includes the word devel developer. Um, how quickly or at all is it at all changing your target group of of people that you want to reach? Because I mean historically, yes, it was very much a developer role and
7:37codeex you first think of developers as well.
7:40Um, but I guess we for us and also in the market see that like the roles are quickly merging between the different um yeah uh roles that we had before. Uh how quickly do you see that in for example your events like did the target group already change the visitors change?
7:58Oh absolutely. I think like there's there's two dimensions right? There's like um what is the definition of a developer and that definition is kind of rapidly changing and evolving. Uh and then there's also like um who has now access to these tools and superpowers that would be uh otherwise limited to them in the past. And so uh for us like we um we don't want to draw like a very sharp line between like this binary decision like oh you're either a developer or
8:28you're not. Um now what we've noticed is that we always want to serve of course the kind of early adopters the people kind of pushing the frontier of the tools the people who are experimenting with the next model every few weeks you know and try to collect feedback directly from them because they are the taste makers. But at the same time, what's amazing to see is that like in the past year alone now like a product manager is way more technical than they used to be. Now instead of writing a
8:57document to explore an idea with their team, I've seen many PMs at OpenAI included building a prototype and actually reviewing that prototype with the team.
9:05Now a designer can even test an interaction instead of just staying in Figma or at OpenI. What we've noticed too and the whole reason why we relaunched uh Chad GPT as one app combined with Codex in a a couple months ago is because we had noticed the finance team, the legal team, the sales team, they were all on codeex instead of having like the need for an engineer to build a dashboard, they could suddenly like build a live dashboard by themselves. Or um if they wanted to like maintain that dashboard and deploy it
9:35every day automatically, they could build that with codeex. for a salesperson instead of having a deck for a customer, they could actually build a full-on interactive experience for that customer. Um, so that's been amazing to see how many more people have this like magic and this like superpower at their disposal and they're all becoming more technical. Um, and that gives us like a better feedback loop with the broader community. Um, and you could even argue that at some point we should almost
10:03rename our team like because you know we go beyond developers to with with tools like dots if we talk about them at some point like you know you have proactive agents kind of like helping you with your goals both at work and in life. And so we go far beyond developers, but we still think it's fundamental to obsess about the developers because at the end of the day, the engineers are still kind of the early uh adopters of that technology and we want to make sure that like we're at the frontier with them listening to their feedback, making the product really great and if they love
10:33it, turns out I think the people right after them will also have a great time. And when you say you don't want to like have a hard differentiation between um like developers and non-developers, is there a differentiation between people who build with Open AI and people who build on Open AI? So what I mean by that is there's obviously people who use all these tools that you've just mentioned, right? Dots, codecs, sites, whatever.
10:59And then [snorts] there's people who just want to integrate with your API and your models. And is that two audiences for you or do you see them merging as well?
11:08We see the interplay happening a lot more in real time. You know, in fact, if you rewind like three and a half years ago when I was joining OpenAI, all we had for developers at the time was an API and we said, "Hey, we have Chad GPT and but if you want to take these models and bring them to your application, you can take this API for the model and you'll be able to do things like structured outputs. you'll be able to like um make your first agents and um
11:37you know at the time we talked about rag and fine-tuning a lot but that was kind of the API that we offered but over time now that you have tools like codeex you can actually build software in like any kind of software even if it's not an AI powered feature but if it's like actually anything that you want to build but assisted with codex and AI and we've seen this interplay now going hand in hand the idea of like I'm starting building an app with codex X whether it's an iPhone app, an Android app, maybe some web applications or
12:06something. I'm iterating quickly, but of course I want that experience to be awesome for my customers too. And I want them to have the same feeling as I have when I'm using Codeex. So I'm I talk to a lot of developers who like I want to bring speech to speech to my app. I want to bring to my app. I want to bring all of these capabilities that I have in Codeex, but I want those in my app, too.
12:28And we've noticed that already. And in fact, it's very timely because um just at Dev Paris this week uh what uh what I announced was uh this new launch where you can now have as part of your chatbt subscription. You now have API credits as well added on top.
12:46And we've also launched an OpenAI developers plugins for Codeex. So all in all, what that means is that I'm starting building something with Codex. I want to have the most frictionless experience to bring these capabilities.
12:59I don't have to worry about setting up an AI key because Codex can do that for me automatically from Codex. I don't have to worry about putting a card on file again at the beginning because I have credits automatically to start building right away. And guess what? I don't even have to read docs because I can just ask codeex, hey like I would like you to bring uh speech to speech here with GPT live and when my customer is uploading these pictures of their uh home for instance, I want you to
13:27generate an image of what the furniture could look like if you're building like a home decor uh application for instance. So uh back to your question, we really see this into play more and more and we actually want to facilitate that. uh whether you are an API uh customer on OpenAI or a codeex builder, we want that uh to be very like seamless to go uh between the two. And maybe one last question regarding that audience.
13:54In the past, addressing developers was probably not cheap but straightforward, right? There is like conferences, there is uh content sites where you could advertise, there is events that you could host yourself. Sooner or later, you would somehow reach these developers. And now that like product people come in, designers come in, everyday people from the street come in and just like start building, how do you actually reach those people and how do you make them aware of the fact that by some definition they are developers as
14:22well and they should feel addressed by what you do?
14:25Yeah. No, it's fascinating and it's a it's a great question, right? Because these tools are very much blurring lines. Um that's why on on our team like we've been spending a lot of time trying to educate uh people with video content uh different kind of formats like written guides on what you can accomplish with the platform. The fact that for instance you can not just ask codeex to build an app but then say hey actually deploy it on the internet and all of that you can have a chip site up
14:55and running or of course if you have like a versell or cloudflare account it can use that as well like all of these things like um chadp sites actually having like a full-on uh database u you know synchronization websocket support all of these things is quite magical because you don't have to worry about infra as Um so yeah it's a rapidly evol evol evolving field. We want to make sure that people know about these capabilities that are being added to the
15:22product. uh but meanwhile that's also why we still care a lot about uh DX because I think uh developer experience is also itself evolving and what might have been for stripe like let's design the best API and the best SDK we can uh still very much is relevant but also expands to other things like well now I'm fully expecting that my customers are using coding agents and tools to assist them in building software and so
15:52that should be The obsession we now have about developer experience. It's like is it frictionless to ask the coding agent to build something? Will it find the right primitives, the right model, the right abstractions? Um, and we're effectively designing now both for humans and their agents uh to make sure that they can on board very seamlessly.
16:12And in that sense, you're in a very privileged position, right? because you also maintain the coding agent that most of these people will use to actually build the products and then integrate again with open AAI. What would your message be for every other platform out there that doesn't host its own coding agent that doesn't bring its own harness that is like reliant on other coding agents to pick them up and to kind of understand like Stripe for example right we mentioned Stripe here before so what is your message to people like Stripe on
16:41how they should be doing developer experience in the age of coding agents yeah um well I think one one uh thing that's been amazing to observe in my view has been that the platform platforms that had the best DX for humans tend to also be the platform that have the best DX for agents already. Uh you mentioned Stripe. Stripe is doing fantastic work but also all the years of crafting great abstractions for humans have actually paid off now even for agents. Uh you see um products like
17:10Superbase for instance like you know agents actually have a great time using with Superbase even though they designed these things before. My message to them would also be like try to keep on building these great abstractions but try to see what um tools do agents need to do their best work. So for instance if you have an MCP server that's even better because all of the sudden like uh the the agent can can uh access that directly. If your docs are consumed by
17:38humans, well, that's already a an a good enough step because the agent will be able to figure out, but if you're able to serve your docs for agents specifically in a more efficient manner, that can make them even more effective.
17:52Um, and then trying to offer the tools too for like things like WebMCP for instance. Imagine you're building not just an app for humans, but if you want the app you're building to also uh be well used by agents. Well, sure enough, agents like Codex have amazing computer use and act as humans. But if you integrate initiatives like WebMCP, then even the apps that your customers build with your interfaces can have this
18:21amazing way to work with agents. So we really think about the whole flow both for the building side uh side of things but also for the user interacting with the end product. Um and so for us for instance the journey that we've taken is like you know we've been the openi we've been building the openi developers plugin uh for codex I I recommend like most platforms to also do such thing because then you can just package up skills and functionalities right there.
18:50And so now if you're using codeex or another coding agent um it's much easier to integrate with your API or your platform because you have this plug-in that's like one click away and I as a human don't have to read every API ref parameter anymore because my coding agent will do that. Do you anticipate a change towards technologies that are maybe less friendly to humans but more optimized for agentic workflows? For example, in the past, APIs basically
19:18always meant REST APIs, right? They're easy to read. They're stateless. Um, they're easy to document. Um, but there's people out there arguing that technologies like GraphQL, for example, are way more efficient. They're just harder to use, which is why hardly anyone out there actually implements proper GraphQL APIs. But now that I don't need to read these APIs and these instructions anymore and my agent and my machine take care of that, will we maybe see a shift towards technologies like that?
19:46It could. Yeah, cuz like the, you know, GraphQL was always fascinating but a bit hard to reason about because you had to specify the field you needed, the field you did not need and things like this.
19:55But the agent could quickly uh figure this out for you. So that's definitely one way. Um, I think the same question goes for SDKs in some ways, right? SDKs were initially meant for humans to abstract away the complexity of interfacing with REST. Like for instance that every REST API request I'm doing, I want to also make sure that there is like some uh retries built in. In the case of Stripe, it's important the payment goes only once and so there was like the item potency key concept and
20:24things like this. You could one on one side you could argue like well coding agents could just write more code at the same time the SDKs can still be interesting there to package that functionality and if you're actively reviewing the code made by an agent it's also nice to see that code quite short and easy to digest when the SDK is officially hiding some of that complexity but sure enough I think like there are um over time uh we're going to read less code and we're going to trust
20:53agents more and more and I could see a world where GraphQL and and some other techniques that maybe the human developers were kind of like bearish on um could be great for agents.
21:09Yeah, at that point I always wonder like what happens or are new technologies still developed? I mean, GraphQL, for example, was developed in a sense that humans uh interact with it and have an easier way uh for it. And they I somehow imagine like is all of this infrastructure that we have and that we use just come to a complete stop now.
21:28And that's just the current state of that agents know because there's not really need for new technologies like that anybody comes up and says, "Okay, I have a new GraphQL." Because the adoption would be very hard in the future as well.
21:42I think it depends. I think there's always going to be like uh you know products that um could make agents even more effective. I think like uh you see like we we try to come up with like something neutral for agents MD to kind of package up the instructions from any coding agent in one place. Same thing for WebMCP. I think WebMCP tried to like you know provide a net new interface for for agents. Uh we've also worked with the the the folks who came up with MCP in order to introduce like UI components
22:11in MCP. That's how we build like the Chad GPT apps for instance and the Codex plugins. You have like um these UI primitives that you can expose not just for us but for any kind of client of that technology. So you can see that even if the models are progressing and the coding agents are being more and more effective, there are still ways also to kind of uh bring some standardization across the different agents and always kind of polishing that.
22:40I want to touch on um a kind of new set of challenges that you might have experienced in developer experience. Um so in the past when working in DevX you most often interacted with developers right or at least with people who were from the same domain. So when you talked to them about like what are the constraints within that you operate you could explain to them why maybe certain features aren't available or why they might take some more time. And I feel
23:08like AI has kind of opened Pandora's box here because people just assume that these machines are like super intelligent and almost to a human scale.
23:17And now when they approach you and ask for other features or why doesn't this work or why does it behave in this and that way how do you explain to them the intricacies of like well this is an LLM, this is a transformer. It's it's not a human after all. It feels like a human very much and often times but you still work within certain constraints and how do you communicate that?
23:39Yeah. Um it's challenging right because uh even the the capabilities are changing so frequently now with with the space of AI but ultimately I think that's where the we're kind of reinventing also the the the work of the developer right what what you used to spend time on with all this toil all of these migrations now you're spending that time being more creative be taking an idea and like actually turn it into a real product and
24:07I think like even if you're a fantastic PM and you're able to prototype something and you can drive like excitement among your team to build a feature because you can actually interact with the prototype. It doesn't mean necessarily that the PM should actually go deploy that feature themselves with, you know, little understanding of the infrastructure components and things like this. But at least you can start speaking like more of the same language, right? instead of having a PM that speaks documents and
24:36the engineers that speak code, at least we're getting closer and closer to like operating on shared surfaces. But you're right in terms of uh the challenge of like, okay, why why did the coding agent behave like this? I told it to deploy my app. Why is it not working at scale for like 100 million users for instance?
24:53Well, there's there's different parts there to the story. You have to have an infrastructure to support that, you know, things like that. But um I think that's also part of our um role on you know developer education and trying to understand like uh how do we teach more users about all of these tools and when you zoom out on the number of users of Chad GP today like we're super fortunate that we have like a billion and 200 million like 1.2 billion effectively people using Chad GP every week. I'm
25:22like, how many of those have not had their kind of like AGI moment feeling yet of like, wow, I I had no idea I could actually do that by myself. Like, I can actually. And so, we have so much more potential to kind of lift up everyone with the power of these tools and be like, "Wow, I can actually be very creative here and turn my ideas into something tangible." Um, so I think to me that's that that's the exciting part. uh and goes back to also like the lines blurring between technical and
25:51nontechnical people.
25:53Talking about lines like where do you draw a line within OpenAI when you talk about developer experiences or is all what is userf facing in some way uh developer experience? I mean we we mentioned uh dots for example talked about them in the podcast uh um often. um is that a product you are involved with and where you also talk about developer experience or is it just somewhere outside your space? Yeah. No, it's a great question because even that
26:22for us is blurring a little bit internally, right? By definition, dots is not for just developers because it's this idea that you can give anyone uh using chip or codex a proactive agent that's always on that's very capable that's doing things for them. Uh even on this trip my dad has been amazing almost every single day doing something I would not have perceived like the model is really amazing like you know you land at the airport in a different city and it tells you in three sentences like
26:50something critical that you have to know for like I don't know your route to the hotel or the venue for deaf day or something that's quite magical but what's even more interesting and where my team is involved is that well dot is for everyone it's not just for developers it's an aentic product but as an engineer or as a developer it's actually also So giving you superpower because now you can start thinking of a dot as like a very high agency senior engineer that I can actually delegate
27:17task to. So I can be on a plane noticing a tweet from you guys about let's say like an issue in codeex and I can be teching my dot now and I can say hey can you look into this? I think there's an issue that was flagged by Dennis like over here. uh and it will actually spin up codex task to look at the codebase introspect the issue prepare the PR ship the PR and be like all right looked at
27:46the logs found out the issue this was alluded to this and that kind of customer prepared the PR prepared the fix it's now in review uh do you want me to look at it ping someone do you want me to merge it and that's kind of like my dot acting as an engineer right so it's this interesting thing where like even though the dot is for everyone it will also transform form the way we work as engineers because now you can think of your team as the humans around you but also their dots as well in the
28:14picture to be able to take on the work for them take on the work for yourself.
28:18So internally we have a team working on docs dots obviously like product engineering research marketing and all of these functions but we're also very well plugged in with them to to help them educate the world especially with the more technical use cases of dots where I think in my view like dots and codecs are going to start even blurring a lot more when you start not just prompting codecs to create a feature but your dot is
28:47starting to proactively suggest things. It's like, hey, I've looked overnight at this feedback from customers. There's one theme that keeps coming back, which is this feature is not working the way they anticipated. So, I just went ahead and propose the first version of that feature for you. What do you think? It's like, wow, this is magical. This is already like thinking three, four steps ahead for me.
29:10So, yeah, can't wait for everyone in Europe to have access to this. Well, we certainly do have a quite a big number of dots in our uh select uh channels already here, right? And also of course because like business enterprise accounts um can already access it and then like hopefully in a few weeks everyone uh even consumers will have one.
29:30Yeah. But also looking forward to these I don't know uh as far as I know it's planned to have like enterprise dots as well, right? That can serve us in in specific roles for example monitoring and so on. I think that's that's going to be really interesting as well.
29:45So yeah, I think it's going to Yeah, it's going to also change the developer experience. You could imagine having on the developer experience side of things. Uh even on call being reinvented because the first on call page is actually a dot that's able to actually decide what's going on and ask a human to wake up when the thing is ready, you know.
30:06So the kind of scenario you're outlining here is is very or feels very futuristic to some people out there, right? I feel like we see a certain fragmentation in the developer space where some people are very like AI positive AI first.
30:23Let's try all the things. Let's early adopt everything we can and see where it gets us. And then there are some people who either by like company decisions, regulatory decisions, financial decisions, kind of feel like left out from much of this. And how does that impact your work where you basically want to share like all the goods with everyone, but there is a huge part of your audience that is not directly accessible or perceptible to that.
30:51Yeah. I mean like um the adoption rate goes um slightly differently from from like either like company size to company size, industry by industry or even sometimes country. But what we've noticed is that the positive here is that unlike any previous technology change is being adopted across the board by everyone much faster on average, right? Like for instance, like in the banking sector, how many years did it take for banks to have a mobile app? For
31:20some of them, it took a decade to have an iPhone app uh after the iPhone came out. Now, what we're seeing across the board is like adoption of AI is much much faster than any previous tech. Now, you're right that like some startups and digital native companies, they are all reinventing their workflows with AI every week. For some others, they're still kind of like earlier in the adoption phase, kind of like um concerned maybe by like IT safety, security, deployment or cost. And we're
31:48very sensitive to all of these chatter.
31:51That's why we're here to help them adopt. And I think sometimes like what I've noticed by all the companies I've spent time with is that success for AI adoption to me comes from two things that have to be true. One is like each company needs to have like one person at the highest level that gives that freedom and that top down mandate to be like hey we're going to reinvent the way we work on this company with AI and giving that sense of direction and
32:20freedom that it is going to happen. It also needs though like people that are curious that are passionate that are trying things because you're not going to get everything right on day one. You have to re reinvent the way you work with your engineers, the way [snorts] you submit your PRs, the way you review each other's code, the the kind of sensitivity to risk you have before merging to production depending on the industry you're in and so on. You need this bottoms up of like passionate AI
32:48engineers that are kind of trying things out. And I think if you have both, you're absolutely going to succeed. Because the challenge is like if you um if you don't have the top down mandate, the the few engineers who are passionate might not have enough leeway to make change happen.
33:04But if you don't have the passionate, curious people and only a top down mandate, you're not going to try enough things and test different tools and actually be ready to reinvent the way you work. And so you're going to need both. Uh but what I also noticed is that when I share or we share the stories of how we use our own technology at OpenAI that actually inspires people a lot because like just two weeks ago for instance I had a meeting with um a large company from Europe and they were asking
33:33me like okay well you're clearly writing more code sure we hear you you're writing more PRs but let's be honest here do you feel actually more productive or not at OpenAI? I was [snorts] like, well, let me give you some examples. Like, I mean, one of the biggest challenges we had recently was that chadgypt.com was a codebase that we started roughly four years ago now, right? Like before the launch of Chad GPT and [snorts] you can imagine it's like millions of lines of code and by
34:02the time you start four years ago, you're not exactly sure where you're headed. You have no idea about the exact sequence of model capabilities. But meanwhile, we've seen this outstanding momentum of the chat GPT codeex kind of desktop app this year. That codebase is much more modern was made for like agentic delegation.
34:20And we knew we had to do something about these two code bases merging. But if we had tried to do this like 3 four months ago, this was effectively impossible to do with AI and we would probably have like taken like a year or so with our great engineers to do that change. It turns out with the help of Astra and Ultraast we were able to get this done in less than a month and this is done.
34:43It's now it's now in production and we've also done this right before dev day in the midst of everything else that we had to build. And when I tell people like well if you don't believe this is like productivity it's millions of lines of code of very large code bases being merged and now deployed in production and it worked. We've done that in less than a month. Could not have been done without Astra and Ultraast. And they start thinking about their migration projects that they've been putting off because it's always something that you want to put off because you always have
35:12more important features to build. The best engineers are needed for that migration, but they're also much more needed for the other uh more uh business important features to build. And so when you start telling these stories, I think it really helps for them to reconsider, wow, okay, so we need to take these things seriously because if we had considered some tools 6 months ago, the status has completely changed. Like what you can now accomplish versus 6
35:41months ago is drastically different.
35:45Given that you already touched on model capabilities, I have a question about that because a few months, years ago maybe, I was at a conference and I heard someone else from another lab, we don't want to mention names here, but I heard someone else speak and what they said was basically we could stop research into future models right now and we would still see months and years of new products being introduced that only just start to leverage what we already have. because the products that we see out there today
36:14don't even begin to kind of like fully utilize the models that we have today now at like approaching the end of 2026.
36:23Would you agree with that or do you think we've gotten better at utilizing the models that we have? And is the the gap between utilization products and lab capabilities closing or expanding?
36:36Yeah, it's a great question. It's a feeling I have as well because um on one side of the of the picture you can see there's safety that's paramount for companies like OpenAI. We care like immensely about that. This is our first priority. This is why there's this like conversation in the public about like pacing the frontier of research, doing it the right way, pausing a training run if need be because of that safety
37:05approach. But mean in the meantime you also see on the other side on the product side you see the like all of the amounts of ships that we've had at dev day in San Francisco but also what the industry overall is able to accomplish their velocity the number of product experiences that they're able to deliver for their customers and that feel that that pace is only accelerating right um and I believe that's still true that like if you take a model like Astra I
37:34mean the incredible cap capabilities that are latent in there. I'm sure there are so many more product experiences we could still build today even if those models were not changing. But meanwhile, what is also very interesting is that when you take the very best early adopters of models, I'm always surprised by how quickly they get used to this novelty and the next frontier of capabilities. Right? when when when Astra came out, this was this
38:02outstanding moment where like the community for instance realized, wow, it's so incredible at 3D modeling. And then for a week or two, we saw everyone experiment with Blender and all of these like 3D worlds, which was amazing to watch. And then like, you know, this has been what, five, six weeks now since Astra, and people are ready for the next one. They're like, okay, let's bring the next one. We want to see what the next Filipino frontier looks like. And so I'm also surprised by those early adopters
38:30and how quickly they're able to kind of poke at the model to find what it's able to do and discover new things that sometimes we were not aware of like this this like 3D uh capabilities that were emerging. Um but yes at the same time you could pause everything and there are so many latent capabilities in these models that we have not fully grasped yet. I was seeing a tweet I think just this morning about like multiplayer AI for instance and how we interact with
38:59each other with our own dots or equivalent. I mean like this is not quite just AI capabilities for that to happen. They're there. They're already there. But it's all about like reinventing the product, the UI, the UX. How do we work together? When does it make sense for my dot to write to you or where is it going to be like poorly perceived? because my dot should be telling something to your dot instead.
39:24And like all of these little UI UX workflows to figure out that's all product questions that have to be, you know, reimagined with trial and error a bunch of times with the current capabilities we have. So I think there's like really both uh happening at the same time and even if we pause or rather we pace the frontier because we're never going to pause the progress is very rapid and will continue to be rapid but even if you pace the research you will continue to see an acceleration of what
39:53the products can deliver.
39:56So we're approaching the end of our time here and before I ask my last question I want to ask Dennis if he has any questions remaining.
40:05No. Um I mean I mean yes and no I think we could talk endlessly about those topics because everything is interesting. Uh but yeah no I think we got some got some great insights.
40:16Thank you [snorts] for having me.
40:17Okay then my last question would be this. Um from what I understand you've been with OpenAI for like 3 years now.
40:25And you already mentioned in the very beginning like how little of developer experience there actually was when you first started and how much has changed since. What are some of the remaining opportunities that you still see for improvement in developer experience?
40:42Huh, that's a great question. So much um ultimately I think developer experience is like everything that needs to happen for developers to have this like magic feeling when they're using our tools, right? And to me um maybe in the past developer experience meant this very precise playbook of like API design SDK doc sample applications demos you
41:11know all of that had to be ticked you know and we've tried to do as great as possible for all of that playbook in the in the era of stripe but now [snorts] it's like everything that um takes your idea into a product. So that's why even to me ultra fast is part of the developer experience, right? If I'm pairing with the model and if it's a frontier model and I'm in the flow, I'm feeling like I want to iterate on something. I'm not sure what the end state state should look like, but I have
41:38this nebulous idea. I need the model to pair with me. Well, now working with Ultraast, this part of having this amazing DX, it's like, wow, I I can in just a few seconds, I see something appear on my screen. I want that to be like now the way I interact when I'm in the flow. Uh but sometimes where I have to do a complete rewrite of a back end, I want to delegate that to COX in the cloud. I no longer want to have my laptop open when I walk around for 4 hours. This should be completely seamless in the cloud. I can check on
42:08that later from my phone. I don't need to worry about any of this. The harness, that's the Codex harness that's open source. We can let uh developers introspect it, look at it. But it's also the same harness we run locally on your Codex as in the cloud and and giving Codex cloud its own computer. That's I think what we've reimagined recently.
42:29We're in the early phases of that. And I think that's kind of also part of the future of DX. It's like maybe my be best development machine in the future is just my phone because I have Codex in the cloud with the same harness, same tool, same context, same code base and it has its own computer in the cloud to check its own work. That's also part of the DX, right? And then on top of that, it's the proactivity of the dots. The dots actually being an extension of me, being the teammate that I have always on
42:59in the cloud, working with me and for me on these applications I'm building, coming up with the new ideas, making the market research about what competitors are building. And so to me, it's it's fascinating to see that the DX is like basically the model, the speed at which we're delivering them. also the cost and the token efficiency at which we're delivering them because imagine we have this frontier outstanding models but they're expensive to run and you're not
43:26able to afford them that's impactful for the development experience because it means the DX cannot be the same for everyone. So we're trying to also push on token efficiency for that reason. And [snorts] so it's all of the playbook we knew with the coding agent friendliness on top of that to make sure that like everything that was in your way before writing code by hand and all of this that's all gone because it's taken for granted. Then the coding agents will take care of most of that for you. But it's also making sure that you can work
43:56effectively either by yourself when you're in the flow or with your team.
44:00Everything going from like how do you do codec codeex code review or how do you harden your infrastructure with codeex security like all of these pieces to me have to all fit together in the sense of like as a developer it should be as seamless as possible to not just take my idea into a product that my customers love but also making sure that I have all of these tools to make it better every day more secure every day more cost effective every day and all of
44:28that. So, so much more progress to do on each of those, but we can start seeing the convergence of all of these bricks together now.
44:37Well, uh, thank you very much. Thank you for taking the time. Uh, a bummer that we didn't get to talk about Jolly OS and Jollybook. I saw that on your LinkedIn and I didn't even realize that was you, too. So, maybe we need to have a follow-up at some point in the future.
44:50We could we could talk about that one for a long time. But um you know it's funny because like you now that you mentioned it I've actually been reflecting on that recently because I've worked so for context for the audience briefly this was my first company back 20 years ago. I was building an operating system for the cloud and at the time like HTML 5 was very nent. Um the Chromebooks did not exist yet but this was this idea that you could buy an inexpensive laptop and just boot it to the cloud and the cloud would
45:18effectively be like where your apps live and where your data lives and everything.
45:23Kids ask your parents about netbooks.
45:26Yeah. Yeah. Yeah. Uh netbooks are Yeah.
45:29We we used to uh they were like all the rage for maybe 2 3 years because the iPad did not exist. Multi-touch tablets were not really a thing yet. But we had these small laptops for 300 or so euros.
45:44So very inexpensive. They were very small form factor but they were shipping mostly with Windows XP which was already no longer maintained at the time. And so clearly there was the need for a new OS for this new generation of cloud. But uh the reason I was reflecting on that recently is because when you when you see the progress with AI and the fact that like anyone can create software so quickly, but also this era of like personal software like imagine you have
46:12an idea for an app for yourself. Well, you would never build such an app back in the day because it would be like too much uh work hiring some developers and such. But now like a few prompts of pairing with the model like Sol or Astra and you can just get your app up and running just for yourself. [snorts] So the idea of an OS that's kind of reinvented for each individual is now insight right you you could re reimagine this like jolly cloud Julio OS version
46:40again uh but with AI now but yeah we can save that for the for our next discussion.
46:48We'll do okay thank you so much. Thank you for taking the time today, Omen. It was really nice talking to you. Um, and maybe we can repeat this at next year's uh developer event in whatever cities we'll find you next.
47:01Amazing. Well, thank you so much uh for having me. Sorry we missed each other in Berlin, but I'm glad we could uh we could do this today. Thank Thanks so much and thanks everyone for listening.
47:10Thank you. Thank you everyone and bye-bye.
47:15[music] [music] [music]