0:06Welcome. How's it going? Good.
0:07Doing great. How are you all?
0:09So nice to see you all. [cheering] Isn't this a great event? I feel like it's just it's so happy.
0:15Yeah. It's good vibes.
0:16Yeah. So, I want to start this conversation off with the question that I think is on everybody's minds, which is a year ago we uh were told that AI might make PMs obsolete [laughter] and now I feel like the the vibes are that everyone needs to become a PM. You know, one of the things I love about getting to talk to both of you is just by being an anthropic and working with the models in the way that you do, you have a little bit of like a sense of
0:45where the future is going, especially for different job roles. And I'd really love for for both of you to talk to us about what is a great PM at Enthropic look like. Is the PM role going away? Is it changing? And if so, how? I think the job of product is always to be a bridge between the real problems that people have in the world and the technology that you can use to solve it. And like that's always been true. And I think a thing that is weird right now is that it used to be that the technology would
1:15change like every 10 years or 5 years and now it feels like it changes like every two months. And that's really hard. That's just like a like as a human I feel like it's really hard to learn how to deal with that sort of change.
1:28And so what I always think about is like the technology side is changing but the human side like the human side doesn't change that fast. We have the same problems. And so what I always think about is like how can I be obsessed with like solving the human problems and then stay super fresheyed about how the technology works? like how can I basically throw away most of what I know about what used to work to solve this problem and then try again like knowing
1:58everything I do about the problem.
1:59And so I feel like that part hasn't changed. Um, what I think has changed is it's almost like rewinding the clock.
2:07Like when I back when the dinosaurs were in the earth and I started in product, like we didn't know, we didn't have definitions. Like we just didn't we didn't know what the job was. And so you were just like a general problem solver and you'd like walk around and try to solve a problem until you like ran into a wall and then you would ask somebody or figure it out. And I feel like that's what we can do now is go back to a bunch of the roles blurring, but like you just keep going and try to figure out what's going to work and then when you hit a
2:36wall, you have all these different sorts of scaffolding.
2:40Yeah, that makes sense. Mike, anything from you?
2:42I think for me, I had a real like um sort of realization moment recently and so earlier beginning of the year shifted to an IC role. So, I'm mostly building.
2:50Um, and we were getting close to shipping something. And recently, uh, and one of the like PM leads works with Ammy pulled me aside and was like, "Mike, I know you love to build. This project's going well. Like, it's looking really exciting. Like, we really need APM on this project." I was like, "I don't know. Like, do we need a PM? Like, Claude's got it. Like, we've got a lot going on." And she's like, "No, we really need a PM." And it like such a reminder like once they joined like all of the things and all the glue and all of the connective tissue and all of the things that were about to get dropped if I had not done that would that would
3:18have happened. And it was like just such a cool eye opening like before and after. Um so much so that the week later I pinged her and I was like cat you were 100% right. Um [laughter] and thank you.
3:27And it was just and I think about all the things that that encompassed. was, you know, actually bringing along, we're a company that serves everybody from, you know, proumers all the way to really large enterprises. Like, have you brought all of that contingency along?
3:41Right? When uh we need to go and figure out how to enable our customer success team to like talk about the changes as questions come up in real time. Has somebody thought about that? Has somebody looped in safeguards? It's obviously very very important. Are people getting kept like kept on track?
3:53All of these things. And nobody has infinite capacity even with you know the assistance of these tools. So to be able to have like call it what you will but somebody playing that role of making sure that everything is going to go well everything is going to get connected.
4:06we've kept the end user needs in mind, Tommy's point, and that the builders who are like heads down and are like, you know, in full-on cloud mode can keep doing that without having to then pop out and do the very different activity, which is how do you actually make sure all these things go well is like I think very essential. And I think all that's happened now is that because we can move faster, that role is like you have to be operationally excellent in that role in a way that like even beyond what you had to be before. But it definitely has not gone away. In fact, I have felt this like renewed like yes, this is a like if you're recruiting for a team and you
4:35need people to wear these hats, this hat is increasingly important.
4:38Can I ask you a serious question though?
4:41So, only serious questions. Serious. This is serious.
4:48Why didn't you just have Fable do it?
4:50[laughter] Like, what is And I presumably there's some like Fable 7 you've got access to that we don't have access to yet. So, like what is that person doing that you couldn't just have Fable do? I mean it is interesting like the we have as you can imagine we like run on cloud tag and we have a lot of like you know cloud provided connective tissue too and it does catch things where you say hey like there's another thing going on in this other part of the organization that you should be aware of you know like we found it via the slash search. So there is that sort of piece as well. I don't think claude is a convenor yet. I had
5:19this um I gave a talk a couple weeks ago on like what's the role of the human in the like post super powerful AI age and there's like a bunch of archetypes I was thinking about but one of it was like the convenor like somebody still needs to bring the clouds and the people together to like get the work done and so I think that is the role that I cla organizational pull or the scheduling ability I've yet this is going to be a moment Dan when claude schedules a meeting with me or for me where I'm like who scheduled this meeting it's like oh did I thought you two should talk chat that has not happened yet [laughter] but not because it can't I guess we just
5:48like turned on that setting, you know.
5:51I [laughter] stay tuned.
5:57I um I think we're also living in a world where like the tools are really good and you can build a lot of things and you can build them really fast. In some ways that just like expands the universe of what is possible to build and you still need to have really good judgment about like what to build based on sometimes pretty limited information cuz it feels like everything's moving really fast. So you have to like figure out what to build and then I think you in some ways you have to be even more
6:25relentless about making sure it happens because there's so many forking paths that you could take. Yeah.
6:36I have an idea of what we need to do and I have the feedback loop to the people who are going to use it and I know what the human problems are and I'm just going to go for it.
6:43And I think that sort of relentlessness and like judgment in the face of ambiguity is is really important.
6:49That is definitely something that I have found, you know, every time there's a shift, every time new models come out, it's it's sort of like you're looking out on the ocean, you're like, what's beyond the horizon? And then you get to fast forward beyond the horizon and what do you see? Well, there's more stuff.
7:02There's like a whole new landscape, you know? It's not it's not like a utopia and it's not like horrible. It's like there's just new lots of new things to discover and navigate. And I think the same is true for just models in general.
7:13when we were talking Ammy in and in in planning for this conversation, one of the things that you said and you've had a long career in product management um but prior to anthropic is that you spent decades learning how to answer specific kinds of questions as a product leader that you now no longer need to answer.
7:34And so I'm curious about okay, what are the skills that you spent such a long time building that you actually don't feel like are that useful anymore? and what are the things that you're leaning on now more and then how does that how does that filter down into the team and your expectations for people that you hire?
7:49I think it's really true like I spent a really long time trying to understand how to put myself in the mindset of a user and how something would fit into their lives and like what we should build because it was so expensive to build and ship something. So, you know, literally this would happen where people would have reviews and what we'd talk about is like, am I where should we put this button, you know, and we would talk about the interaction pattern and where like someone's finger would go on their phone and what situation they they would be in where they pulled their phone out
8:18of their pocket and what they wanted. And I feel like I tried really hard to get good at that at that mix of like product and design. And I just like I don't think anyone's ever going to ask me that question again.
8:29That's a fable. That's a fable one.
8:30[laughter] That's in the past. Like now it's actually faster to just build three versions and try them out and be like, "Oh yeah, this works for this one." So a thing that I prided myself on is like not actually that useful anymore. And I do think that's really hard just in terms of identity. We talk about the innovator's dilemma when it comes to companies where like you're so good at one thing, you keep on trying to do it even when there's this other thing that maybe you should try doing, but you can't because you're already so good at one thing. And I feel like that's also
9:00true on a personal level where I sometimes feel like, oh, I know what I'm good at. I'm just going to keep on trying to do that even when there might be a different way to do it that might be more important in the future. And so that's what I just encourage everyone to think about. Like when you think about that product bridge, part of the technology changing every two months is you have to throw away what you knew about yourself in the job using that technology and just try something different. And most of the time I don't
9:28know if I'm going to be any good at it, you know, like I've never worked at a model company. Like I just this is not a place where I expertise and I I just kind of I feel like every couple of months I have to constantly be like, "All right, well, let's try this other thing and see if I can figure it out."
9:42You know, what are the characteristics of someone that is able to do that well? And what have you had to develop in order to do that well?
9:48I mean, I think it comes back to the the sorts of things we were talking about. I think like raising your ceiling on tolerance for change and being willing to embrace change is super important.
9:59Like so I think adaptability I mean I'm sure this is not new. Like adaptability is just such a key thing given how rapidly the world is changing. Um, I think that sense of like judgment and relentlessness as we talked about, those are like the things that I keep coming back to and really what I try to think about for the team is like I think it's important to talk about those things because otherwise it can feel really lonely like you're the one who has to constantly adapt but actually everybody
10:28has to constantly adapt and I think it's hard on all of us. But I also talk about trying to frame the chaos so it doesn't feel as taxing, you know, like I think I think when there's times of uncertainty sometimes it's tempting to like try to map out here's exactly what's going to happen.
10:47Like here's exactly the products we're going to build. Here's exactly how my career is going to go, you know, like and that's a way to exert control. And I just I I think that locks you out of trying all the new stuff. And so what we have to do is like frame the chaos so it feels safe and it feels plausible to engage with it. But that's where like a bunch of the magic and productivity is going to be. So just like talking about it, making it safe, acknowledging that there's also emotions around all of this
11:15and it's not easy, but it is a way to just participate and understand and work with the future. I think that's something we talk about a lot. I think there's an organizational leadership question that I think Amy you do really well which is like you can have the chaos but you also need very clear leadership and DRI ship within some areas. So like you can have a bunch of people sort of exploring something very frontier but having a clear like this is the person who's ultimately going to make the call whether this is you know the thing to pursue or not. In labs we call them leads or like bet we have call
11:44them bets within labs. So the bet lead is a really important role and they are the DRRi for saying like I think we should double down on this or like actually we should wind this down. this team needs more people, this team needs less. And like that role is even more important because we're moving so quickly, because things are shifting really rapidly to still have that like we're all lost together, but somebody's got the like, you know, the pen on like what what's the next most important thing we can do to derisk, to understand, to learn, to move things forward, too.
12:09We've talked a lot uh already so far about uh the psychology and the skills of being a product leader. Um, one other place that's changing a lot is what you actually build and how you think about like the features and the functionality in particular because we're moving from a world where you'd expect only humans to use your software to a world where uh humans and agents are using software.
12:34Sometimes it's delegated, sometimes it's fully collaborative. And I know Mike, you've sort of been at the center of figuring out what is what does agentnative type software look like? And I think you have a lot of thoughts on where that is going. So, can you talk to us about the uh h how you think about what to build, what goes into a product versus an agent and uh and what's what's the overlap?
12:56I love this question and like I maybe there's like a timeline to look at where V1 or like er 1 was great, we've got this AI thing, maybe it belongs in like a sidebar or like a little mini window and you can ask questions and it's kind of disconnected and if you know maybe it helps you with some customer support questions. Um and then sort of greater integrations where maybe there are whole features that are AI powered and then this movement towards being more agent native and I think every like literally wrote the book on it which is and the way I for that the whole year where I was working a lot in this idea of like
13:25agent native architectures I would basically feed their like write up on this and then create a skill out of it because I was really good encapsulation of it was the idea of like everything a human could do an agent should be able to do as well and very few products actually get this right. I think a lot of our products like don't quite get this right yet. But when you do that, all of a sudden sort of emergent behavior gets unlocked where you know the agent can sort of piece together things that weren't pieced together before or can proactively offer um different ways of doing things. I think we're still on the journey of making that good, but I think the next stage
13:54that I've been thinking about a lot is when the interfaces actually become something that is malleable by the agent too. And we've talked about mal malleable software for years. I feel like it's actually now like really manifesting and becoming real. And um maybe back to the sort of role of product and connecting to this question, one of the things that we have, we have like a pretty complex thing that we're trying to ship right now. And it's got at least four independent orc streams.
14:17And the way we've operationalized it is we've had Claude both sort of keep an eye on how everything is going, but also create the UI by which first the TPM and then the entire team understands what is happening in that project. And what's really cool about that is that if you didn't like that particular display, you're not bound to like however it was determined to be built either by some internal acceleration team or some uh sort of third party. You can change it um and you can change it along with cloud and it's just been a very cool thing to see internally. Like one of the biggest shifts in how we've worked in
14:46the last year I think at Enthropic is so much of the software we interact with is like built maintained and iterated on by claude. of course reacting to all the things that are happening in the organization but very much like changing all the time and so that leads to all sorts of other interesting questions like who can update the data right is it just the humans clicking through is it also clawed in the background how do you have like provenence of data it opens up some like interesting questions there as well but I feel like that is the piece like the both the trend and the thing I'm most inspired by right now is how do we make that the way almost all of our
15:15software operates and of course you still want building blocks and predictability and design systems and you don't want you want to frame the cats you want to like make it not absolutely insane, but for the given task, for that given project, for even that given user potentially, what is the way in which we pull in the right um ways and make that really feel both personal and like extremely useful because of that and how does someone think about maybe you know you're not in anthropic and you're you're have you have a SAS app that's scaled and you're thinking about okay, do I put an agent in it? Do I make it open and agent native so that cloud
15:45can be in it? like how do how do you think about those choices and what a company should care about versus uh leave to leave to a model company like Anthropic to to plug in? Yeah, I think a lot of it is really building the right primitives into the product that almost like an infrastructural layer and you can kind of tell like I I always poke at the whenever I'm using new products um I'll ask it to do something that you know a human could do but you know might be required somebody to think through or architect the thing where everything goes through some shared plumbing that like both agents and like whatever sort
16:14of rest API you've built can go through as well and you can tell when they've been thought through that way and it's I have a lot of empathy because if you've been building an application for 20 years just bolting that on is difficult.
16:25So I think is a journey that a lot of companies are going on. But once you get it right, I think that question becomes you can just evolve it over time, right?
16:31You can start with, okay, at first I'm just going to have maybe more of the side paint thing. I don't I'm not ready to blow up my whole UI. I'm not ready to create fully generative UIs, but maybe you start experimenting with like, all right, there's also maybe a personal landing page that can be more malleable and is using the same primitives. You're not inventing a whole new sort of infrastructure underneath. So I think that you can evolve your product along the way, but you have like you can tell when things feel super bolted on and like like that's you know feeling like it's not a native part of the product.
16:57So Ammy I'm curious we're hearing we can go all the way to you know agent native malleable software but uh anthropic you have to serve this whole spectrum of people. there's this uh challenge of pushing the frontier and also you know some people just want to use cloud to like chat or you've you you have big enterprise contracts where you can't just like rip out their whole interface and then say hey it's malleable like you know on on in one day or maybe you can't I don't know but I assume you can't so
17:27how do you balance [snorts] uh in terms of thinking about product pushing the frontier and creating a a consistent experience for users that they can understand and bringing along the people who are maybe not going to try all the new things on day one.
17:42I think it's a really hard problem. Um I mean our approach is to just try to meet everyone where they are. I think we're just so early in like understanding what works. It's not like stuff is stable and we can just build the right answer. It's like we're all trying to explore together. And part of the everything changing every couple months is like the models change, but also like the way people use products and the models changes and we kind of constantly have to update for that. And so a thing that I think about is like when we're just
18:11exploring a a product like like Mike's talking about, it's okay to just try stuff like we just have to try a bunch of things because we haven't actually explored the frontier. And then when we figure out what works, then we have to like make it make sense for all the people who want something that feels more stable and complete. Um, I think this is a really tough trade-off because, you know, like raised in like the the product like world that I was, I was all about like simplicity and
18:41reliability and total predictability because I think that helps people build like a muscle memory and an intuition.
18:47But I also think the reality is like we just don't know yet what a lot of things should look like. And so I actually encourage the team to just like take a lot of shots on goal. You know, it's okay to have things that feel somewhat overlapping or duplicative as long as when we figure them out. When we hit product market fit, we go back and we reinccorporate into something that works for everyone. Because I feel like the fail state is you decide from the beginning like based on a theory of exactly what needs to happen and you
19:17like over constrain a product. And I would much rather have like five overlapping products that all work in different ways than like one overly constrained one that we never even gave a chance to actually like meet the market. And how do you and I definitely see this we we do this at every but like I see this all around the industry just doing lots of parallel experiments and seeing which ones one one works and then starting to consolidate as you start to get uh get signal. How do you do that in a way that doesn't feel confusing or
19:47incoherent or you know if I'm working on the third team doing this thing like it's can be maybe demotivating like how do you do that? Well I think there's a couple of things. one like I think when we were just having this conversation this morning internally where you know there was like two potential like product directions and one has like a team with very high conviction going after and the other one doesn't really and like it feels very manufactured to be like we should explore this other idea cuz we think it's good but nobody's like really calling for it then that's like that kind of B team feeling that I think I've
20:15never been like successful tasking a team to go like take a product area that they're not excited about like the product is going to suck at the end of that process right um so like you want you want the like excitement but if there are two three different teams that have exciting directions and yeah like let that let that explore. I think I I think we've made a lot of progress on even in the last six months is like having the underlying infrastructure to support all of that. So, we inventoried this. It feels like 6 months ago. It's probably only three, but like there was a point where like chat and co-work had different memory systems and different
20:44MCP and different um ways of storing files. And it was like, okay, like for anybody to experiment on top of the surface, they're going to have a really hard time because if they built a third thing that then doesn't share memory with any of those, for example, it's going to feel disconnected and like almost disadvantaged from the beginning.
20:59So, a lot of the work, you know, we have like a foundations team that has like been just thinking about like you should be able to get your memory wherever you are within our products. And like it sounds like a simple thing but then it's very clearly you pull back the thing like oh it's complicated for these eight reasons but it's worth solving that and solving it in a different team than like your labsy innovation or like kind of frontier product teams because then like that those experiments can feel complimentary even if they are overlapping too.
21:20I also think like this sounds so simple but I also think it's just really important to talk about it and just name like hey if there's three different bets sometimes it can feel demoralizing or like one of you is not set up for success. That's okay. That's the world we're living in. And so what we're asking you to do is come along this journey with us. We don't know the answer. We don't think anyone knows the answer. We're all going to discover it.
21:42And we're all fundamentally on the same team. And so we will just like learn from whatever works. And just like saying stuff like that out loud, I think is really important because everybody is like trying to figure everything out in the dark.
21:55What do you look for to know that something's starting to work?
21:58I mean, I think a bunch of that is kind of the normal product market fits. Like do people use it? Do they like it? Do they come back? Do they talk about it?
22:04Do they get value? Like again going back to like the goal of product has remained the same. It's like to solve problems. Thinking about does this actually solve a problem in a way that people can use and use easily and feel good about using it comes back to the basics for that.
22:19One of the interesting things that feels challenging to me about that is sometimes a thing looks like it doesn't work and the new model comes out and then it really starts working and you don't want to like throw it away too early but you also don't want to be too far ahead. So how do you do that? Like one of the things we do in labs sometimes is just park a project. Like when we built we built the first like computer use product inside Anthropic in 2024. Um and it was so bad because the models just weren't there yet. And you would be you know our first thesis was well it's not going to be good at a bunch of automation but it might be good about education. Um, so you'd be like,
22:48"All right, uh, how do I, you know, change this, you know, thing in some complex software? Like, how do I, you know, do this really complex thing in Photoshop?" And it would get there, but it would get there in like the very like early agent cloud way, which was like this, did that work? Nope, that didn't work. I got to close this. It was like 20 minutes later like, I did it. You're like, I learned nothing. I learn, you know, so like it was neither good for automation nor education. But we parked it and then we would basically just try it with every new model. And we actually, the way we would try it is we actually just had it like running in an eval harness. And the way we actually learned that 37 was like a like a leap
23:18in computer use was like just seeing it suddenly took over and then we actually looked at the transcript. We're like wow it's succeeding more often than it's not like this is a real moment. So it is really valuable like we see it as a win in labs if what you did was all if all your project did is show that the models aren't ready for it yet but you're able to now like externalize that as an eval or sort of like create some uh connectivity to the research team like that's great because it means that 2 3 4 5 6 months from now we'll be able to revisit and hopefully we've learned something along the way um as well but I
23:48think you need you need to build early so that you can even have that intuition.
23:51I definitely agree. I've been calling this like it's it's sort there's a sort of you can get capability blind if you try something and then you don't ever try it again or you don't make an eval and you're like oh the models are just never going to do this and then it just turns out a model comes out like in 3 months and it just completely chang it it improves in ways you never would have thought it would. I'm I'm actually curious if you're here how many of you have used Fable? Raise your hand. Oh, a lot of people are using Okay, this is great. This is a good What about Astra?
24:18Fewer. Interesting. Okay, so this is great. you guys are you guys are on the you guys are on the edge. This is this is very helpful because one of the things I find a lot is I talk to someone and they're like, "Oh, it doesn't do this." And I'm like, "Have you used the latest model?" And they're like, "No, I used like Sonnet 45 or what?" Like whatever. And uh that that really changes things.
24:36I think that's also part of why it's so important to be willing to throw away everything, you know.
24:40Even though it's hard because then otherwise you just like think that the last thing that happened is going to be true forever. One one big question that I I see you guys working on from afar and I see this happening at OpenAI 2 and and we run into it a little bit is when you're doing all these parallel experiments and one starts to really work then you have to like push it back into your actual product and there are several different ways to do it. It's like okay you can make a tab but then you have a bunch of different tabs and there's all all the things like only work in like in slightly different ways
25:10and that's kind of ugly. Or you can have it just be a separate app and you never merge them. There's all these different things that I don't think anyone has really come to this is the answer, this is how you do it. But it seems like a critical problem is how to take those parallel experiments and turn it into a coherent product. How have you what have you learned and how do you think about it?
25:29I don't know what you're talking about.
25:30[laughter] Um, you're right. I don't think we've fully figured it out. But I think some of it comes back to what Mike was talking about of making sure that we're building primitives so that our products feel like the way they should feel, you know, when you're using them. Like what does a user expect? The user expects to walk up to something and like hopefully not have to think about it too much.
25:52Like and right now it's easy to just transfer a bunch of cognitive load to the user and be like here's some tools like why don't you choose? And so one of the things we that that Mike's talking about is like how do we make sure that we build primitives so the right system knows you. So a UI that feels like really accessible all the time. And I think all of that is a bit iterative as we see how people react.
26:17I think overall like if I had to bet like there's some lessons you'll learn and some that are are real. Like the thing that we always saw at Instagram it's like there's a very strong power law in tab usage. It makes sense. every single product right it's like main feed was like 80% of the use right and you can make explore better and maybe you would go from 10 to 15% but it was like main feed that is where you land that is your big thing so I think that is will always be the journey that we're all on which is like what is the main what is the main tab like what is that experience how do you still make that great and then luckily it can be an
26:46entry point to a lot of other downstream interesting experiences but yeah I think half of the job is saying no to another thing in the sidebar or another tab or another thing and figuring out Can this be rolled in once we've proven it out but not killing it so early. So this is the entire I think art of product development I think as things are moving this quickly.
27:04All right, we've got time for one last question which I'd love to hear from both of you which is if if we are on this stage here next year and Lenny invites us back. I don't know if he'll invite me back. I've been cursing too much but he'll invite you two back. What what do you expect will be different and and how will being a PM have changed? I hope it just like that that people feel like they can just do much more like they can just go and build in whatever way they want. They can get answers whenever they they want. They can start businesses and like and we just see a
27:34lot more like small teams and solo empowerment and we we just we start to get the uplift of like what that means for everyone else. Like I think it I think it's always like builders magnify impact for a bunch more people who will never like use the tools directly but who can benefit from people who do build things. And so I just I hope that's just way easier.
27:56Yeah. I think the biggest one for me is you know right now like I often talk about like the gap between what the models are capable of and like what most folks are using them for. Not through any fault of their own. It's on us. Like we got to build the products to make that possible. And like that was a hard thing last thing 24. was even harder last year. It's getting even harder now.
28:13And so like the biggest thing I want to do is close I think it's very democratizing if you can close that gap because it means it's not just your most like clawpilled software engineer that has you know like this fully working multi-ogenic setup. It's also the person that is doing like a bunch of research in parallel. It's also the person that's trying to manage their business but they feel like they've unlocked it and we've brought them along the way in terms of how do you like understand the moving parts, check in at the right places, do long horizon work but not in a way that feels disempowering. That's the art.
28:40Like if we succeed a year from now, that'll be closer.
28:43Amazing. Thank you so much and Mike to [applause] see you.
