0:06Hey everyone, it's so great to be here and I love being here to get to talk about one of my favorite things ever, being a PM, product manager, and builder. And what's pretty incredible is that it's never been so interesting to be building products. And after 20 years almost of building, particularly in consumers, by the way, it's very upsetting to be able to say out loud.
0:29Um, but after almost 20 years of building, I've been fortunate to build all kinds of different products over my career. I've built startups and I've also helped lead teams and products through really big changes like Instagram and Google search where now billions of users are using products that our teams have created over the years. Also have built some things that zero people use today, by the way.
0:51Absolutely zero. And what I've learned throughout this process is that there is a little bit of a pattern of the kinds of things that we've done to make the products really great. Sometimes it works out, sometimes it doesn't. But there has been a repeatable pattern that I've seen emerge from the things that have done really well and the things that have not. And that's really what I'm here to share today uh and in part to all of you. There's lots of things to talk about, but I wanted to give you just a little bit of extra background on me. So I spent some years at Instagram
1:18where my team has built out Instagram stories and shipped it originally and built all of the experiences around it.
1:25We have reels, feed ranking and direct messaging. So a whole bunch of products that came out of that era. Uh and then now at Google search where I am working through a brand new uh experience with the team to help think through what is Google search in the AI era so that you can truly ask anything on your mind and get the most helpful information imaginable across everything whether it's videos or audio text and you can tap into all of the knowledge of the of
1:53the web and of Google and connect the information that's out there. But today I'm not talking about the transformation of product, although that is something I really deeply enjoy. I'm talking about the craft of product. What does it take to build? What does it take to have a product that people love and to shape that and make that? And how do you do that in the age of AI that ultimately from my opinion any one of you can do now with just a handful of agents or kind of AI assistants which took us
2:21years and whole teams of people to do um in the process here. Let's uh just mention one thing I wanted to say around the the role of PM before we get started and that is really the the fundamental shift that we're all feeling is that we used to be I think as PMs a big part of our value was around getting stuff done was around organizing creating energy and momentum and being almost like a project manager and a product manager and I think in some cases that's still true but really if in the world today
2:51almost anything you can think of can get built it's more clear to me ever that the actual true value of PM is actually around judging. It's around taste. It's around doing something extremely well.
3:03And I think what's something I've thought about a lot is that people say the PM's main craft is that you do lots of things pretty well. And I think that is somewhat true. But I think above all else, actually the craft of PM is one simple thing that you should be the absolute best at. And that is decision-m. It's about making decisions and you should be students of what it takes to make great decisions. And it's a humbling thing because most decisions, many decisions are wrong. And these are just the things that I've learned over
3:33the years that have helped me and the teams make decisions that have led to some of the great experiences that are now out there today. There's kind of a book that's been unwritten uh in my mind that I've been kind of thinking about um in three chapters around the process that you know I've used and teams have used to build and I'm here to share that today really for the first time in this format. So chapter one is understanding people deeply. It all starts with people obviously and you can figure out lots of things that you can build but it comes
4:01back to the very root of what are the fundamental needs of a person. And I love the book um Clayton Christensen wrote competing against luck. I'm sure many of you have read it and are probably read the jobs to be done framework which I really do love. The thought is that we don't use products, we hire them to do things for us. And we have to know what those things are. And it's best articulated how to extract those kinds of insights through a story.
4:24And so actually um a team that we had uh at Instagram, we did a offsite with the people who created the jobs to be done framework. And we actually went through this process. So you can all think about this with me. What was the last major purchase that you made? So quickly think about it. And for me it was a bed. So here are a bunch of beds. And then what you're supposed to do, and the book actually talks about how to do this, we've really internalized this with our process, is to put yourself back into that moment. And we're talking to a person, ask them details about their
4:54life. What was it like that day? Where were you? What were you reading? Who were you with? What were you wearing?
4:58Were you with your kids? Were you by yourself? Were you in a store? Were you online? And you really deeply put yourself in the position of that user.
5:04For me, looking at the sea of beds with my wife. And basically, we were kind of trying different things out. And at the end of the day, three or four felt great, but my wife weirdly turned to me and said, "Can you do me a favor and jump on this?" Basically, uh she was like, "Do a barrel roll." I was like, "What?" Uh all right. And so I did like a huge flop, I guess. And she was like, "We're getting this bed right now. Thank you. We're done." And I was like, "What?
5:27This is so weird." She's like, "No, the main thing that I need is a bed that will not wake me up if you like move."
5:33And if you think about it, that's like a little offensive, but also I was like, "Okay, it's real." And if I was a PM for beds and I might be like classically thinking quick what's the road map for beds right you might be thinking okay uh how about cooling everyone's doing cooling like yeah cooling sure or like eco uh or just pure price like those things do matter but for us the thing that made the decision on that day is a concept of transference and if you didn't talk to me and you didn't learn that the PM the team they would never know that and so that's what it means to
6:02discover the job to be done for someone and how products things customers hire ire what your services do, your products do to help them. And very quickly, just talking two quick examples in the digital world, one from Google. Early in our AI experiences, we realized that people actually came for inspirational journeys. But two years ago, chat bots were only text. So, we were one of the first teams to look at this need and say, how do you get inspired by what to buy or or what kind of uh couch to get
6:31through text? How do you describe a couch to someone? You can't. And so we were one of the earliest teams to really invest in state-of-the-art multimodal understanding, retrieval, and knowledge systems. So you could actually ask about a couch, get an incredible visual response, and do multi-turn conversation with it to say, "Oh, I actually want emerald green, or I want fuzzy, like what does fuzzy mean?" Or, "I like that throw pillow. How does it create attention on the throw pillow?" And do a turn and then now do a grid of throw pillows. Like these were really hard things many years ago. And obviously you
6:59see them now in many places, but that was a core need and a core insight that we saw. And then the last example was very early in our AI mode exploration which is a way that you can use Google search now to ask anything and have a conversation with Google was people came and it had all this incredible information like here this is a neighborhood and like a food question but it was missing something. It was missing something that people were coming to Google for and were expecting, which was the knowledge and the accuracy and the experience of actually seeing
7:29the places you go, seeing them on the map, getting the Google level authority of the star rating, the closing time, what kind of food it is, how expensive it is, it's completely missing from the product. And when we brought that together so that now you can talk to Google, ask about food and then get this rich experience that brings all the goodness of seeing the food and clicking on an entity now and actually seeing the description that you can trust. That was the magical product and it's still one of the most popular things on the product today based on thumbs up ratings and other things that we follow. But
7:58again, back to a human insight of the why behind they were using the experience and what caused them to use it in the first place. And so AI can actually change a lot of this and a lot of the talk today is how AI can apply to these. I think on this one I'm still thinking about some of these uh myself.
8:15But one of the things I'm excited to try more is how you can actually take the transcript data of these user interviews and actually have it specifically source jobs to be done and scale that much more. Another idea I've been thinking about um is actually an agent that can interview people with the exact methodology that I just described so that it can really ask a user more at scale to talk about why they're using the product they're using and what those underlying problems were. So that is understanding people. Next up you have a seed, you have something you're working
8:44towards. The next step diagnosing root causes. So the way I think about it is you have this initial vision of this problem you're solving for people. this initial understanding of what people want and then you make a thing and that thing is usually not great and most of the things that I've ever been a part of building started not great but everything did really um and what's incredible about this process is by having a clear and analytical view of the exact problems that are in your
9:13product space ranking them in priority and understanding them deeply and then actually fixing them and then asking it recursively again and really looping through you basically are doing your own training process, kind of like an epoch of model training a little bit. And each update you do with this change list gets you closer to the market fit you want until you have that fit and then it works. And there's two stories I wanted to kind of articulate here, both uh from Instagram. Uh one was when we originally launched stories, it it actually did take off. It was one of the earliest things that worked pretty well out of
9:43the box, but um we had many internal versions where stories were fit into the feed that didn't work great. So there was a ton of um headache to get to the the final version, but one thing that was particularly clear was that not everyone was adopting it or felt comfortable sharing. And really the whole premise was, hey, this is an opportunity to share your life throughout the day. Why aren't you doing this? And the process that we followed was pretty simple. We asked a very clear root cause of why you were not able to do the thing that was our goal. So it's kind of the inverse. The first one's
10:11like, why did you do the thing? Now you have a thing. It's like, well, why aren't you doing the thing? And it turns out that if you asked that question, people answered. Think of all the leaf nodes like a cloud behind me of all of the answers people had. And then if you were to group them by theme, it turned out that the top one when you sorted them was something around audience problems more or less. Like my ex is on it, my teacher's on it, my aunt is on it, my sister's on it, I'm not going to like just put a goofy thing on like people are on it. And so what we realized in seeing that and then what we
10:40did was we then did a very large quantitative survey. thousands of people taking that as an input and then we were able to quantify the actual weights across them. And what we figured out was that if we couldn't overcome that burden, there's no creative tool.
10:52There's no like interesting product idea that would work. And so the team actually spent 2 years cycling through different ideas in the space until we came across close friends and it ended up working. And there were all kinds of weird permutations in this like multiverse scenario where like it's called favorites and it also has on your profile a back door that you can like see a like a private version of your profile and you can also post to your feed under that kind of close friends label. Super confusing. No one understood it. Like why would I put this
11:20like very raw photo in the middle of like a glitzy feed of all this stuff that's happening. The only thing that worked was within stories. So we cut out the entire product, relaunched it as a stories experience and it worked. Reals was the second example. We had we had a a launch for reals in Brazil and we thought at the time that if you're doing goofy things and dances and posting them, why would you want them to live on your profile for everyone to see forever, which made sense and the team felt that way. So, we launched it. Huge failure. So, it turns out that if you
11:50invest a bunch of time doing an awesome dance, you don't want it to go away in a day, which was an ephemeral format. It's very obvious in hindsight. But the the bigger insight was the people who were posting them, even if they were young, you know, people, they wanted to be entertainers. They wanted to make businesses out of their content and they wanted it to live on and go viral. And so we did a massive update. We talked about this change list where now it's its own format. Instead of being ephemeral, it lives like forever and it can be something your whole school sees.
12:17Instead of the opposite like, oh, I don't want my whole school to see it.
12:20It's like, no, that's actually what you want. You want everyone to get the joy from this product. And then obviously that take has taken off and done quite well over the years. So two really clear examples um of how that's happened and with AI this is actually possible to do a lot easier now using internal tools and I think things that you can all do every day and we use you know internally Google anti-gravity but we have a system that effectively lets users that have opted into this. They want to talk to us more and they've decided that they're they're comfortable sharing as they're
12:49using Google products with us. And so what will happen on the side is we will then ask them, hey, what did you think of this? Did it where did it miss the mark and they will put comments in. And so imagine this huge library of feedback from people now that's available. Well, now the model can take that and can both take qualitative feedback like above.
13:06This isn't this is a way a shopping example for backpacks. And it's like it's awesome that you knew about each backpack and it's awesome that you knew which was larger and the sizing because Google has all this canonical information, but you should have asked me about my son and the height. like that's a pretty key thing or it's going to be too heavy for my kid. And it was so obvious that when you stack rank them, this was a very common kind of feedback. And so this concept of being collaborative and in and creating a conversation so you can actually help the the person make progress with to the ultimate objective because they don't
13:36actually know all the things that they need to know to answer. It was absolutely critical to getting the user to vote and say this was awesome. And so we invested a ton of energy in this and it's been one of the biggest things we've done in terms of engagement, in terms of usage, in terms of value and helpfulness, you know, that we've seen out of the product. So that's kind of chapter two, uh, which is around understanding the root causes. Now, if you're lucky, you have a great seed idea, you've done this process of finding the market fit by go looping through, seeing your problems, analyzing them rigorously, and fixing them
14:05aggressively. Maybe you get to the stage now where the product is ready for prime time. You're going all out. You're going to all your customers, you're sending the blast email. And really, this is about the fit and finish and final elements of the product, which is around craft. And I think this is often overlooked, but basically the way I think about craft is the feeling that I think everyone has a general sense, and I believe people can detect whether the creators of a product cared about the product that they created basically. And there's two things that I think drive
14:33this. One is does the product work perfectly and is there any user pain basically in that experience? And the second is how does it make me feel? Do I feel something? Joy, excitement, using it. And so I'm going to talk about each real quick. Um, on the pain side, it's really not just about obsessing over it and you all being power users of your own product, but actually how to scale that now with AI, which is really incredible. And so I actually built a tool that I'm sharing here. This is a real tool that I built um internally
15:02using the anti-gravity system to basically look at all of the potential questions we might ask. So, here's a sample here on the side. Use the product as an agent. So, like literally like go to Google, type in a question, go to the AI, ask a follow-up, and then record that experience, screenshot it, and then evaluate what you see based on a really comprehensive rubric. And what we found out was that the model actually was incredibly capable to look at things like, hey, there's a math question, or this is like a school homework thing. Is
15:31it doing a great job rendering math and latte? Or is it, you know, emitting weird uh characters? Or if you ask about bioluminescence, is it adding a visual tray, which is what the response should have if you're asking about something visual or is it just text? And you can go through and you not only can you get to the best response, but you can see breakages really easy. And the team easily and the team has now invested in a bunch of agents that can go figure out when stuff is just broken or offspec and then increasingly fix it itself. and you
16:00have this feedback loop and where you used to have maybe a QA team, you you may need a PM or you might need people on your team who are obsessively using every single part of the product. I think many of those days are now over where we can scale this process much more elegantly for everyone. And then the last thing I'm going to say is around you know delight and really sparking joy. And this is a little experience. This is what the Google search box looks like now. We we reinvented this the search box after 20 years announced it at IO what an AI based search box should be. It should grow with you. It should be able to
16:30upload things and have context around files, around videos, around photos.
16:34What what should that be like? And and how daunting of an experience is that?
16:38And what we did was we came up with a a system that said, okay, we wanted Google to still feel Google and but we wanted it to be this upgrade that felt like AI is here. It's there's AI inside that's helping you. And so there's this concept of this kind of gradient of color that really represents the AI kind of power behind the product. And you can actually see a couple nice details here that the design team brought together. One was actually in the cursor. If you watch it blinking, it's cycling through the different Google colors. And I can't tell you how many times I've seen this
17:08pop up on X that like someone noticed this and and and felt like posting it.
17:12It just blows me away every time. Or if you tap in the search box, it's kind of excited. It like jumps out kind of and then loads the ring. It's like, "Ooh, like we're getting ready to load. Like we're we're here." You know, it has this kind of feeling and it actually has real feeling with haptic. So you actually see imagine all that coming together as an experience and it's just one small example of just by using the product and through motion through color through feel through touch you know and the intentionality of the experience you communicate the humanity behind the product and you give it a soul and I think many people overlook this you know
17:41part of the the product creative process and then really when you bring this kind of all together this is kind of the the wrap here I think that this really means that you can have a a playbook for what it means to be a PM in this new you know AI frontier here and it's you have to still do all the same things like a lot of this is not new but it's it's possible in a totally new light with AI so you have to understand people deeply get at their core needs and problems that they're feeling you have to be rigorous and analytical at sorting out what are the things I need to fix in
18:11order to get the product to win and to work for people against that objective and I need to build it so that it's awesome and it's simple to use and it works great and it makes people feel awesome using it and that's really the playbook and I think that now it's exciting that the craft of PM is so critical. I think it's never been more important time to be a PM and to do product and to be a builder. And I think the skill of making great decisions that can lead to great products has also never been, you know, more important. So hopefully all of you can take a little bit of something. It's just one way.
18:39There's so many ways to build products.
18:41You probably heard 10 other ones today that were just as valid as this one. You should decide on what makes sense for you, but hopefully you take one thing away um that you can take away with you and to build as well. And hopefully the next time you're thinking about your products, you're spending a little bit less time thinking about, hey, here's cool features kind of out of the blue, or here's some tech we're going to throw at it, or here's a new way we're going to get it in front of people. And you spend more of your time thinking about the actual people and the humans on the other side of the screen that you're building for. Thank you so much.
