0:00You are head of product for Muse. Every PM's dream is to work on the number one app in the App Store.
0:05I've never experienced a product built like this before. We would give Mark our phone and be like, what do you think?
0:09And he'd play around with it and give us feedback.
0:11You have been there from the beginning.
0:13Nat Friedman pinged a few of us. [music] He's like, I'm using Oculot. You guys need to get on it. And I spent all weekend talking to my Oculot. It felt pretty magical with such little direction from me. The more we were using the product, it was like, okay, if we want to build it for our consumer base, it has to be really different. It's going to have to be way [music] safer.
0:30Felt like y'all were way ahead of this, but it took you a long time to launch something.
0:35There's a saying we have internally, trust is a must. If we launch too early and we have a bunch of issues, people will not trust this agent. If you don't trust the agent, you're never going to use it.
0:45The little character Jolly, not something you'd expect from a company like Meta, like a massive company.
0:50Jolly is like our cockroach. It just like never dies. Every few [music] months, somebody would be like, there's no way you're going to launch this product with a stuffed animal at the top. But I think when you're talking [music] to something that feels so personal, the last thing you want to talk to is a corporate logo.
1:04What's next? What's kind of the vision of where this heads?
1:06Right now, it's sort of trapped in an app. [music] Soon, it should be in your glasses, on your charm devices, whatever that might be.
1:12What a time to be alive.
1:14When I was a kid, could you imagine?
1:15Like I had to go up to the TV and turn the TV down to be able to change a channel. And now I am talking about personal super intelligence in your glasses. Mona, thank you so much for being here and welcome to the podcast.
1:32Thank you, Lenny. Thank you for having me.
1:35You are head of product for Muse. You're VP of product at Meta Super Intelligence Labs. Um, Muse has kind of come out of nowhere. I feel like it's I don't know what the numbers are, but it feels like the most successful AI assistant out there right now. Zuck has hinted that it's the fastest growing AI consumer product of all time.
1:57Uh, Uh, you have been there from the beginning. You've also not been on any podcasts. You've also barely posted anything on socials. You've had like five posts on threads, like one LinkedIn post. So this is extra awesome. These are my favorite conversations. People doing amazing work that nobody has really heard from. So I have so many questions.
2:16I want to start with just kind of the origin story of Muse and kind of your story of of of getting started on this work.
2:23What was what were you working on before you joined this team? And then what was kind of the the pitch they made you?
2:29What was kind of the kernel of the idea when they pitched you on taking on this work, joining this team?
2:33I was on the um in the AI org and the superintelligence org uh before I've been I've been working on it for about two years. I've been at Meta for about 10 years. Um and I was mostly working on Meta AI uh which is our assistant product. Um and we were zero basing Meta AI at the time and it's a phrase that we use a lot where we're like, you know, we have something it's not working quite the way that we want it to and you could keep optimizing
3:02it or iterating on it um or you could just start over. Um and so we were doing a lot of that with the primary Meta AI product because the lab was working on this new model which was the first of the Muse Spark series um that we were anticipating and it was awesome. We were having a great time.
3:18And then I think on like Thursday, maybe end of January or beginning of February, Nat Friedman pinged a few of us and he said to us, I'm trying out Open Clau.
3:30It's changing my whole view of what's possible in terms of um personal superintelligence and you know, consumer agents which we had been talking about quite a bit. Um obviously everybody was sort of talking about it in our head. I think I wrote this down actually in like an H1 brief. I was saying that yeah, we'll work on it probably end of 2026 maybe 2027, right? Like that's kind of the time frame when I think that this sort of thing is going to be coming to a head. Um and and yeah, so that's sort of
4:00like where we were at and then so he sends us this message. He's like, I'm using Open Clot. You guys need to get on it. So I think on Friday, I was told my husband, I'm like, I need to go find a Mac Mini. I need to get Open Clot set up. And we looked at the Apple Store.
4:14There were no Mac Minis. So then he was like, I think I have a Raspberry Pi somewhere. So he got his Raspberry Pi. We set it up. It happened to be a weekend. My children are in they're in a ski race program. And so they were gone for most of the weekend skiing.
4:27So it was just me and my Open Clot. And I spent all weekend essentially talking to it. And like, you know, I had at this point, I think my husband came back on Sunday with the kids. And he was like, how's it going? I was like, oh, I gave it access to my email. I gave it access to my calendar. And he was like, what are you doing?
4:44That is crazy. That is so not safe. And I was like, no, no, it's totally fine. I wanted to like, I'm going maximalist. So this is like a Sunday. I had spent the whole weekend talking to to this and setting it up and hooking it up to my home automation, all of this stuff.
4:56And then Monday morning, my kids are going to school. And I'm just really bad at personal email and like just staying on top of all the school newsletters and the portal. I never check it. Like so my children are, you know, like we do our best. Um but they're often not figuring out Spirit Week or anything like that, all of these kind of special things. And and I had gotten a newsletter from my Open Clot that I had, you know, kind of set up. And it was checking all of these different sources. And it sort of pulled together this snapshot of like what to
5:25expect this week, what to expect today. And it was like, I remember looking at it. Now in retrospect, it's like, okay, a lot of people can do that. Even assistants can do that. But at the time, it felt pretty magical and with such little direction from me. I had to, you know, it was like such like such a small prompt that I had written, right? Maybe like a few words.
5:43And the idea that I could kind of suddenly feel really productive and on top of my which I never felt on a Monday morning, right? Particularly in my personal life, was like incredible for me. And I looked at it and I was like like, you know, how many busy parents are there? How many busy people are there?
6:01How many people feel like this, right?
6:03And So anyway, so I pinged Nat and I was like, this is it. I think we got to, you know, I'm in, right? Like, I don't know what I'm in for. Nobody We had never done like an edict of like we're going to build this or anything, but I was like, wherever you want to go with this, I'm I'm going with you. Um and a few of us kind of felt that way as well. And so um and you know, we had a really great team on Meta AI and so it was like totally fine for a few of us to peel off and start to experiment and and see where this concept went. But it
6:31really for the first time felt like this idea that we had been And And Mark actually wrote something about this, you know, before any of this started. And if you go back and read his like, you know, his his writing about personal super intelligence, it's kind of crazy how much was actually coming true um you know, with this harness and this idea of being able to take these models and and making them far more authentic.
6:55So that was It wasn't really a pitch as much as it was like a ping that just ended up consuming me completely. Um and uh and you know, I like there wasn't even like a work stream, it wasn't like a product to join or a team to join. It was kind of like a few of us that decided to see if we could build something similar because what we also were realizing was as the more we were using the product, it was like, okay, well, if we want to build it for our consumer
7:23base, it has to be really different, right? Like, it's going to have to be way safer, it's going to have to be way easier to set up, it 's going to have to be scalable. Like, there are all of these other issues that we started to find in between the magic that we realized, you know, for what we would want to do with a a personal agent, we would really have to make some huge bets and swings.
7:45But because we didn't quite know where that was, like we didn't want to formalize it immediately as like a, you know, an official project and it kind of stayed in this like zone of like trying things out and seeing what worked and prototyping and for a bit. And then at some point we were like, okay, this is a thing. Like we're We're pretty sure that we're going to have a product here. Um and that was when we sort of mobilized proper team around it.
8:08This episode is brought to you by this season presenting sponsor WorkOS. What do OpenAI, Anthropic, Cursor, Replit, Sierra, Clay, and hundreds of other winning companies have in common?
8:19They're all powered by WorkOS. [music] If you're building for enterprises, you have felt the pain of integrating single sign-on, SCIM, RBAC, [music] and audit logs. WorkOS turns those deal blockers into drop-in APIs built for B2B SaaS. [music] And now there's a new kind of user that needs access to your app, agents. AI agents need to register, authenticate, and get permissions just like a person. WorkOS handles it all.
8:45Agent auth to give agents scope, revocable identities, agent registration with auth and D, and [music] airlock to judge every agent action and allow, deny, or escalate to a human. I've noticed that every startup that I'm an investor in that expands [music] upmarket ends up using WorkOS because they're the best. Go to workos.com to make your app enterprise ready today.
9:09One thing that is interesting about this timeline. So when was that NAT ping roughly where he's just like let's start exploring this? Do you remember what it was roughly?
9:16It was like end of January, beginning of February.
9:19Um it was cuz like OpenCloud, right?
9:20That was happening sort of around that time, like January, December, January, I feel like was when all of that was kicking up. So it must have been end of January when we started the conversation. But it wasn't until February when we were like confident that this was something that we wanted to like truly pursue beyond again like us kind of playing around with some concepts.
9:43What's really interesting about this is it felt like y'all were way ahead of this much earlier than all of the other what are now popular AI assistants, but it took you a long time to launch something.
9:55I believe Groq Bot took a couple months from idea to launch. I think um Duet is a similar timeline. Dots is a similar timeline. Took y'all seven months from what I've read from beginning to end. I imagine there were very good reasons to wait that long. Talk about the that decision of not launching it quicker.
10:14First of all, I think there were many moments throughout that period where we were like, I think that we're close, and then something would happen to cause us to like think a bit more.
10:22But I would say like the the things that there's like there were sort of three phases of the work that we did. The first was just like what is this product? What are we building? Right?
10:30And in that phase like we like didn't worry so much about like reliability and scalability and safety and all of that because it was just a few of us using it.
10:38And just that piece of like figuring out there's not a lot of precedent to what we were doing in terms of like like the closest precedent again is like open cloud, which is you know a very very different kind of product in the end. Um but just figuring out how to like run a bunch of virtual machines, right? Was like its own interesting problem to solve. Um I think we agonized a lot about um certain decisions within the product like you know should it be a chat thread? Should it be
11:08more like an assistant? Like some design choices and we can talk a little bit about that. Um but those all kind of came together and relatively quick. Like by spring I think we were like you know okay we've got like a pretty fully functioning end-to-end product.
11:22But you know I think the things that we really ended up spending a lot of time on was reliability, safety, and security. Like those three areas were we just were never like our bar was so high. Um and I remember we had these conversations where we were like well what's the eval bar? And again like because there wasn't a lot of precedent, right? Like we couldn't really compare and the the benchmarks that are out in the industry just didn't feel sufficient to where we wanted to be for a product that's this I mean it's got your personal context, right? Like it can it
11:50can go out and buy things for you. It can you know like it could do things and so it just felt like we needed to hit a totally different bar that we had to kind of create on our own. Um and like spend a lot of team time red teaming. And then like new features would kind of pop up, right? Like in the beginning in the very beginning news didn't have a browser.
12:10Right? So uh but then it did. And then when have a browser, suddenly you have a whole new set of problems that you have to solve, you know, from a security angle.
12:19Um, so as like new features would kind of come in that we felt were critical, it would open up new vectors that we had to make sure were like completely closed off. And so, you know, I yeah, like I think that we basically I mean, it wasn't like we held the product, but it was kind of like we really wanted to make sure that we were quality gated, because there's a saying that we have internally, trust is a must. But really, like we were like in that you know, it's
12:48like if we launch too early and we have an issue like we have a bunch of issues, like that's it, right? People will not trust this agent. And if you don't trust the agent, you're never going to use it.
12:58And so we just felt like we needed to get to that point. And uh and yeah, so I you know, in retrospect, my husband was actually asking me this the other day. He was like, do you wish you had launched a lot earlier because all these other agents started to come out, right, kind of around the same time? And I don't think I'd change anything. Like I really think that we like there was a bar we needed to hit. It took us the time that we needed to get there.
13:21And I'm glad that we did because we like we so far haven't had a catastrophic problem.
13:28Um, and uh and I think we just continue to like you know, the whole industry is so early in this space that there's still just so many unknown unknowns. Um, so we're never going to be perfect. But um but yeah. So that was kind of the thing. Like it was We also did a lot of like dog fooding internally where we had employees use it and you know, we had alpha testers externally using it. So it sort of felt like we were like not launching the product, but we were getting like bigger and bigger groups of
13:56people who were using it and sending us a bunch of feedback. Um, so it did feel like throughout the summer we were starting to get real user feedback uh so that by the end we felt a lot more confident.
14:07You mentioned this phrase, trust is a must. I've heard other phrases uh kind of I don't know, mottos internally that you all use. I want to understand what they're about. One is uh the claw is the law.
14:18[laughter] What's going on there?
14:21Okay, so yeah, that that happened after that initial kind of like you know, first few days when we were using it and this was also a Nat Friedman special when he we saw sort of saw the magic, right? Like of this concept uh we not no one else knew what we were talking about.
14:41Like I think that there is a few people who were using Open Claw, but like the majority of people had never heard of it, had no idea what it was, and it seemed pretty scary, honestly. Like when you did hear about it, you were like, okay, that doesn't seem very safe. Like why are you doing that?
14:52And so um we had to sort of explain like why this mattered to us so much and why this was going to be the thing that we really believed would be the future of the product that we were building of of you know you know AI and personal super intelligence and um so there was this document written, Claw is the Law.
15:11It became a great catch catch phrase. So uh and uh it was also just a way for us to be able to help people understand where we were. Um so that was I forgot about that one. We haven't used that one in a while.
15:23It was kind of like a vision kind of north star of what the idea is.
15:27By the way, I never mentioned as you were talking about Mac Mini, I have my Mac Mini sitting right here where my Open Claw was used to run.
15:33That's so funny. Yeah.
15:34was just getting like get a Mac Mini. Eventually I was like, I should probably upgrade this Raspberry Pi and I got a Mac Mini. They took me some time.
15:42is like it's unnecessary now and this is you know, that was a big debate back then. Will we need these Mac Minis? Are people over buying Mac Minis? And now it's like okay, all these other tools have launched that are much better. Why we don't need these.
15:54So you've talked about Nat. I want I'm curious just the um the roles these folks that we know of have played in the development of Muse. There's Alex Wang who's very out there on socials. There's Nat, Zuck obviously. What's kind of been their kind of roles in the in the development of Muse?
16:10The team was pretty small and I think that um the way like again like I've I like I've at this company for a really long time. I've never experienced a product build like this before. I never worked on a lot of different products here. But this was like the probably the most unusual experience I've had and felt more like my startup days before I joined Meta than anything else.
16:34And I think the part of the reason is because of the team and it was quite small. It was very engineering led, very engineering focused. And part of the reason is the way that we interacted with the leadership team where it kind of felt like they were more of an extend like I don't know. Like typically at any of any large company or any company really like when you're working on a product you sort of have this rhythm with leadership where you have set times to meet with them, you have reviews with them, right? Like they react to the thinking that you have and it's sort of this like very like I don't know. We've been doing
17:04this for years and years and years in business. And like this was completely different because it was so new, it was so like kind of unknown that we you know we were it was like we didn't do formal reviews, right? Like I didn't make any slides. I actually had to make a set of slides in August I think right before the launch and I was like I forgot how to do this.
17:31[laughter] Like I had to go to Google Slides and I'm like oh I forgot how to make slides for people. But we were sort of like we would go into these rooms with them, we would you know like we'd do build a prototype and we'd like hand it around on somebody's phone, right? Like I would like we would give Mark our phone and be like what do you think and play around with it and give us feedback and sometimes we would be brainstorming, right?
17:53And you know like with the three of them or individually with any one of them. But it was like a series of conversations, I would say with them that led to the product the end product that we had.
18:06And often it was like very very small rooms that we would have those conversations so they could be really really casual and like you know I remember with Mark sometimes like we would have like a we would have a calendar invite be like like minutes that we're going to talk about it. Then an hour later, we're still in there chatting away.
18:24And we'd be like, wait a second, I'm pretty sure you're really important and have other things to do. And he's like, no, totally, I I want to talk about this, right? So, um so I thought that was very unusual because that's certainly not how most of my experiences at Meta have been. I think in terms of the roles that they played, like you know they I think they all had imprints in the product that you see right now. So, Nat was obviously like I mean he was basically part of the team, right? I mean he was the team, right? Um and and
18:53in many ways this was his like he was the one that sort of instigated this um this source of magic for us. Uh Nat, I think, is such a privilege to work with. He's obviously a genius, but he has such great passion for the products that he builds and the people who build it.
19:10And he is like really just an engineer's engineer. We would sit in rooms and he would be like, oh yeah, like can you guys pull up the architectural drawing of XYZ, right?
19:19Or when we were like figuring out the sentinel and how to build a sentinel so that, you know, we could create like a really secure way so Muse wasn't had had basically a backstop, right? That was outside of the core Muse agent. Like he was incredibly involved in the design decisions there with our architects and our, you know, security team.
19:38Um and was just like always around for kind of you know, should we do this? Should we do this? Like we're thinking about this. We have this design. I think uh Alex was obviously critical just from a like he was the he's the model, right? Um and what was cool about that story was that the first Muse Spark model that we got was way more agentic than we thought it was going to be.
20:02So it was actually kind of a surprise.
20:03Like we didn't realize we didn't think that it was going to be as good as it was. And so um and I think part like he obviously very quickly realized that like this was also going to be very important. So he you know, the cool thing about what I when I watch Alex is like he's you know, research is very hard to change. It's like a long-term kind of thing. But he has been very like been really good at figuring out how to be nimble in certain areas and being able to prioritize based on not just what, like, the research
20:31team is going to do, but also how, you know, how that fits into the product. And uh and he loves Muse. Like, in the end, he was kind of the one that, like, pushed us out the door. He was like, guys, I think it's ready. Like, just go.
20:44Um and then Mark, you know, he like I said before, like, it was it was the first time I've ever worked with Mark in this regard. Um but he had he had a lot of ways of thinking about what Muse was that ended up defining the end product. And a few stories that I would say there, one is, like, he, you know, in the very beginning, he was, like, the purpose of Muse is to make people's lives better
21:13and to help them achieve their goals.
21:16And I think we all kind of knew that, but we had never explicitly stated it, and we had sort of gotten lost in, like, the task part of Muse, right? It's like, oh, task completion, task completion, task completion, right? Can it, like, make this purchase? Can I do blah blah blah? And, like, he really kind of pulled us out of that and was like, wait, no, but, like, the most important thing is it's alignment, right? It's like knowing what is the user trying to do? Where are they trying to go? What's what are their priorities, right? And how does Muse constantly help them do that? Well, when he said that, we were,
21:45like, okay, wait, like, that actually opens up a a whole new um area that we can start to think about, which is what we started calling dreaming and study, right? So it's like, okay, if Muse knows all of these things about you, well, it's a 24/7 agent, so it can think about you all the time, and it can think about how it's going to help your priorities, and it can make better suggestions for you. And so we started, you know, like, once he kind of, like, opened that door, it was, like, you could see the engineers' eyes light up, and they were, like, oh, we got now we got a lot of ideas about
22:14how Muse can become more proactive and more helpful and all of that. Um and we also created the goals tab, which is, like, a very explicit articulation in the UI of this concept. Um but but it was like a, you know, and so so so that's like one kind of really like obvious example of where he was, you know, he's obviously been thinking about what personal super intelligence means.
22:41And I think the other thing that is great about Mark is that he builds for people, right? So I think a lot of people are building for, you know, businesses or enterprise users or whatever and like because he's like, I mean ultimately a consumer CEO in many ways, right? Like he starts to think a lot about like democratization of these things and how do you bring it to everybody, right? Which means that like we have to worry about scalability in a way that like is very unique to us.
23:06Um, and so uh so a lot of that also influenced the way that we thought about the build process and what we were prioritizing in our roadmap.
23:13What a [clears throat] dream team.
23:14And obviously post launch Alex has been like our best marketing machine ever.
23:19Oh my god. He's there's like there's like this quote from the Bible Bible about how you were made for a time like this. It feels like Alex has hit his stride. He's found his product market fit.
23:29I I remember when the first set of tweets were going out and we were like, we didn't even know he could tweet like that. Like [laughter] we were like, oh man.
23:37Oh man. Uh speaking of that actually the little character Jolly feels like um a not something you'd expect from a company like Meta, like a massive company that's very delightful kind of [laughter] quirky character. Is there any story there in the backstory of Jolly and how that survived almost through the journey?
23:52I was joking to my the designer who has, you know, he's kind of the brainchild of Muse and Jolly. Um, I was like Jolly is like our cockroach. It just like never dies and every few months somebody would be like, there's no way you're going to launch this product with a stuffed animal at the top and we're like totally no way. And then a like a week would go by and we're like we're we're going to keep it in there, right? Yeah, we're going to keep it in there and then like a month would go by and somebody else would come in and be like, wait, there's no way. So basically it started with um
24:21a prototype. So Alex Cornell who's our lead designer made the the prototype of And he had this like one-shot prompt of a creature uh that he put in there.
24:36And I think actually in the beginning when you opened up the app there or the prototype it was like asleep and then it would wake up. Um and it was called Jollybot because he used to refer to his daughter as Jolly Bug. Um and so that's where the name came from.
24:51And it was like totally meant to be temporary, right? And like we sort of assumed at some point that somebody in corporate marketing or somewhere would come back to us and be like you need to put a logo in there or something else, right? But like once we had it up there we were like suddenly we started to develop these like concepts like okay now that I see this avatar all the time well it should be moving more, right? So it's like you know when it's working it's just staring at me, that's weird.
25:20Why don't we put headphones on it and have it work on a laptop, right? Like why don't we start to make these animations? And some of that was just out of like we were bored a little bit and like we were like I don't want to keep staring at this thing so let's do something fun with it.
25:32So we started to like kind of think about that and then we were like oh well then people should have they should be able to make their avatars whatever they wanted to be, right? Like why like you know it's kind of weird that we're all looking at Jolly and like I a different avatar for myself and you want a different avatar for you. And this really kind of gets into this concept that we have about like the Muse is yours and like we wanted to really be yours and not like a meta entity necessarily.
25:57So then like people started to develop their own avatars and so at some point it became like harder and harder to take out Jollybot cuz it was like we had created a whole system [laughter] around this thing. Um but I will say that there was a moment where we thought Jolly was truly dead and there was an alternative kind of like logo that we would put in.
26:18And I would say that the reason like there are so many reasons but actually Mark was one of the reasons where he was like no he really loved the avatar. But I think mostly it's like when you're talking to something that feels so personal the last thing you want to talk to you is a corporate logo.
26:33Right? Like that breaks the spell completely. And um and so that just felt like not the right answer and it felt so uncomfortable for us. So um you know, a bunch of us on the work E team were like over our dead body. And I think everybody around us you know, and and the employee base also loved their you know, they loved their avatars and they loved Jolly and you know, it was so cute. And so uh so in the end I think between Mark and us um and a bunch of others we were
27:03like Jolly stays. I think this was like a week before a launch or something. We were like this is this is what we're going with.
27:10And I don't think we knew how iconic it was going to end up be and partly think thanks to Alex Wang. Um I didn't think we realized how iconic it would be uh at the time. Um so this was sort of like a decision for ourselves because we loved it and then it just ended up being like in my opinion the obvious choice now.
27:31Wow. That is an amazing story. I had no idea. It feels like this became a core part of what made it successful, just the personality and personalization and feeling different. And to your point, this is like something you would expect a company to kill because it's just something it feels not not professional. There's a lesson there.
27:50One, I think just for what I tell some of the PMs at Meta is that um yes, you are part of a giant corporation, but like you have agency and Mark will hear you out, right? Or Nat will hear you out, right? Like they are like you know, it's like there's not I think sometimes you like it would have been very easy for us to just be like yeah, of course corporate logo like you know, accept the state that you're in, right? Uh as like
28:20But that's actually not how the team works. And um and I think that the fact that we were like we love this product, we built this product and we have a strong opinion about this product, right? Like that mattered and people listened to us, right? And so that was like one that I'm like, you guys have agency, you guys should say what you want, like say how you feel.
28:40Maybe people will say no, but like you should try.
28:44And then I think the other one is that um there's always this like tradeoff of like the brand versus you know, it's like how much do you want the brand to show up versus like not or whatever. Um and I think for a lot of conversations that's really important. I think what we were building um like if it's going to be your muse and it's going to be your personal agent, it has to feel like it completely belongs to you. And Meta kind of needs to fade away from that because at the end of the day, the way that we built the agent was the agent
29:13needs to work for you. You are the owner, right? And we will have guard rails and we will have a charter and we will make sure that the agent is abiding by the law, right? And so those kinds of things we will have, but the rest of it really needs to be you.
29:27And so that like setting those principles at a product level helps a lot because then it's like it's not a random rationale. It's like no, we've always been saying this. This has always been the principle. Um so if you take that, then you know, it becomes very hard to imagine a corporate logo in there, right? Because that completely violates the product principle. So I think that was also really important with setting that up at the very beginning and being very clear with everybody around us.
29:53Speaking of Zuck and kind of coming back to this earlier thread, uh Zuck talked about what his favorite use cases from muse was this kind of watching him do MMA and giving him feedback. And I'm curious just I can imagine a CEO coming in, hey, I need to use it for this, and you're like, but this isn't what it's great at.
30:10Uh is there anything there about how you dealt with that sort of maybe unexpected use case?
30:16So actually a lot of our first conversations were just about use cases. Um and like hours and hours about use cases, right? Uh and um you know, it's sometimes funny when you talk to people within the company because they we are not normal, right? Like in so many ways, you know, partly that you were working for a giant tech company and, you know, you're far more AI pilled and agent pilled than anybody else.
30:42And then Mark is obviously like, you know, very much just like, you know, unique uh and some of his problems. But so we spent a lot of time talking about use cases and what we would like talk about was our own personal use case. And because we had, you know, people testing both in we had this like very long uh UXR panel that we started very early on. Actually, that UX panel started I think in March.
31:08And this was internal people or external?
31:10External. So we had external people um that we had recruited. We also had the internal people obviously using it. And we would and the internal people we have a forum. I don't know if you you're familiar we have workplace, right? And it's basically like Facebook groups for internal and people post in there, right? And so we had a feedback group for uh for Mews.
31:31So between those things we were getting all of this feedback about what people were doing and how like what they were using it for.
31:36And we would bring it to Mark and Matt and Alex almost on a weekly basis and like kind of obsess over it, right? And we were like here's what like, you know, here's all of the stories that we've heard. This is what people found magic in, what didn't find magic in, all of that sort of stuff. And so it actually was kind of nice because yes, you're always going to get like someone's like, oh, I MMA story or whatever. But because we were kind of like rooting everybody in these like real life experiences um it never like we didn't actually get thrashed on that front.
32:04What did happen was there were like surprises. So uh I did not think shopping was going to be as big of a use case as it's become. Like I would not have predicted that. I love to shop personally, but I would have never predicted it. I predicted personal productivity, health uh and personal finance were like kind of the three that I was like pretty focused on.
32:28I think a few people were like home automation, right? Um and so like that was kind of like the zone. And then we started to see shopping like pop up a lot more with our dog fooders and it was a complete surprise and so then we were like wait like all of our evals have been over here. We actually need to now focus on shopping and make that a pretty big use case and evals around that and we basically have a use case level evals, right? And then home automation, which we thought was going to be huge
32:59Like it just you know it was like really big during the open cloud days and I think partly because a lot of people who were using it probably also have a lot of home automation to that already is set up, right? But as we were getting more into like jet you know like just like all of these you know our like a broader user base that just didn't pop as much as some of the other use cases. But now shopping is one of our top use cases.
33:21I heard there's an engineer on the team who loves shopping.
33:24Yeah, Christina Wod. She is the head of engineering so she's the so we've got Tarek who is our kind of our tech lead our core architect and then Christine who's our engineering manager and I I am terrified about how much money she spent through Muse.
33:42[laughter] She is she is like it's so funny cuz like we'll wake up in the morning and she'll just send all of her bug reports, right? From shopping use cases and I'm like Christine I need to take your credit card away. Like I need your credit card the engineers need to sleep today.
33:57[laughter] But yeah no she was she was definitely our like number one dog food around that and then I so I have also recently started to shop a lot more and my husband called me this I was you know I was in Menlo Park this past week and he called me this morning and he was like when I've got seven packages out here. What I realized is happening is before I used to bulk order, right? So I would go to like like I was like oh I need to buy sweatpants for my kids. I go to nike.com and I bought like five pairs of sweatpants. But now because I'm having it with Muse it's going to like
34:26lots of different stores. It's picking like five different places to buy from based on my spec, right? And it's actually like much better because I'm getting diversity and like you know whatever. But it also means I am getting five different packages or seven different packages and my husband was like I don't know what's happening right now, but like, you really need to calm it down.
34:46[laughter] He was actually at some point he was like, I just want to talk to your views and give it instructions from my end. And I was like, that's really funny because that does make a lot of sense. It would be, you know, it would be great if you could talk to them.
34:59of look at your partner's perspective on shopping versus I don't know how much time you spend thinking about this, but there's it's such a new world that is emerging in so many ways, but one is this idea of how we shop and how we find stuff. Just, you know, like the world like retail has been built around we'll market to you and convince you to buy this thing and show you ads to buy this thing that we are selling. And now agents are making these decisions. Do you spend any time thinking about just the how this changes
35:27ads, marketing, shopping that agents are now deciding?
35:32People really enjoy being inspired.
35:36People enjoy the act of browsing, right?
35:38Like it is, you know, it's like some things you kind of you're like, okay, like I just want the toilet paper like tomorrow or whatever. But like a lot of things it's like quite nice. Like I I like looking I like you know, I like looking at a bunch of clothes and like outfits and figuring it out.
35:54So I think we're going to have to figure out how to like make both things work, right? Which is like let the user be able to have that inspiration, let the brands or the companies help the user figure out, you know, what the option space is with and have the agent really help like narrow the field or, you know, like make those trade-offs where you're like like sometimes you're just paralyzed. I'm like, I could do this one or this one, and actually I'm just going to buy
36:23nothing because I like couldn't make that decision in that time, right?
36:27Like those are places where the agent's really helpful, right? And it can help you make that that choice. So I think that like we are going to have to find a way for the full use case to exist because I actually don't think people are going to be satisfied if we cut out browsing, right? Like that's going to be sad.
36:44Um, and I think people will like still want, like they still want to do that, right? So anyway, so I I do think that we are very early, which means that what's happening is I don't think anyone was ready.
36:59Like there wasn't like a master plan that was set out, right? Um, on what was going to happen. And that probably is true for any technical revolution that happens or any any revolution that happens, right? And you sort of have to figure it out and ideally you're figuring it out together. So we've been really lucky to have partners, right, who are on the retail space or who are merchants who are excited about this potential future.
37:23They're testing agents out too, right?
37:25They've got business agents and like how cool is that when like a business agent and a consumer agent can communicate with each other and help both sides, right? And like what kind of information can a business agent have that could like be even better for the consumer, right? Like cuz there are some things, right? It's like I have to go to the website, I have to filter, I have to sort, I have to search, right? Like you know, like there are some things where like the agent's going to be very helpful and be able to like really cut down on your time and just get you to
37:54the things that you want. But again, there are some things that I don't think like we shouldn't replace because that's just enjoyment for users and how users make decisions and that's still really important.
38:04I want to go back to something else that I've heard about how you all work and built Muse. I hear that there was uh a agent swarms that uh helped you scale user feedback, helped you figure out what to pay attention to, which uh sounds awesome.
38:19What what's the story there?
38:20Yeah, there are these agents that we like kind of like swarm out into the, you know, into the ether, right? And they're able to kind of like look at all of the conversations that are happening and data that is happening and basically like kind of come back with uh and they can also create synthetic data as well, right? Um, and so like you can have like all of this data and uh uh it was a way for us to be able to basically feed into our
38:51Right? So it's like what's happening out there? What's like what's going well?
38:55What's not going well? Right? We would do these things called like dumps from the swarms, right? Like they would it would like come together with this like giant dump and it was like this like wonderfully formatted articulation of what was happening in the ecosystem in terms of like the you know like like you know what kind of tasks were people doing?
39:16So this was looking at at like metrics analytics and feedback people are sharing themselves themselves.
39:22Yeah, exactly. Um so because what hap Okay, so the the thing that we were really struggling with was um when we were working on Meta AI, most of the conversations were what I would consider to be mostly single turn conversations, right? So people would have like one kind of turn, you'd get a response, maybe you'd have another one, but often you wouldn't.
39:43Um and I think that's changing for all assistants now, but for a while I think that was probably pretty true. And then when we started building this, it was like completely multi-turn, right? Like it was like back and forth, back and forth, back and forth. And we don't have data that looks like that. Like you know cuz in order to build our evals, we need data and often you want data from production, not data that I've like written, right? Um and so we just didn't really have any data. And so in some ways the swarms were critical because there was like a way for us to be able to collect that data and understand what like multi-turn
40:10conversations over long running tasks could look like that weren't about coding, right? Like again like a lot of this stuff existed for like coders or whatever. And this was like you know how do I meal plan for the week or whatever? And so um so they were really critical in terms of like being able to like get a lot of insight and a lot of data for us to be able to feed to like create data sets as well as understanding what was happening. So now actually every time we have a new model that like comes in or
40:39if we're like the other day I was like I think that there like I was spotting some like you know thing that I was like I'm not sure if this is an actual issue or if this is just a me problem, right? And like the engineers were like oh we'll swarm it and figure it out. Swarmed it, and like, oh no, this is like an actual problem that we need to solve, and we're going to get somebody on it, right? And so, it's been very, very useful because it's super fast and very like in comprehensive. Actually, like makes me wonder how we even built products before because it's kind of crazy. Like it's like I feel like I have so much insight now to be able to work off of and it's
41:08You know, before it was like you would look at dashboards of metrics, right? So it was like you would get a dashboard, you saw the metrics, the metrics went down and up and whatever. And now like you still have that, but you also have like examples from production, right? It's like, you know, these like you can get so like focused.
41:27It's like, you know, 10 people have asked about, you know, to shop on this website and got blocked, right? Or something like that, right? And you can get so specific with like this information now to be able to really like put in place target changes or improvements or whatever. But it's like the thing I make all of my PMs do it uh because I think you you know, it creates a lot of empathy. And I think one of the things that the swarms did was when they bring back this data because they're bringing back not just numbers but insights, it creates a lot of empathy for what the
41:57users are going through um and uh and kind of like helps us like feel the pain, so to speak, uh which I think is really, really important for product development. I think if you don't have that empathy and you can't feel the pain, you will never develop good products.
42:10One of my favorite quotes from this podcast is from Nan Yu who is now at OpenAI who was at Linear head of product and you have this line about how your job as a PM is to feel the pain of your customers as much as they do.
42:21Yeah, 100%. No, I I think like maybe the two things that drive me as a PM is um constant anxiety and like the the really the pain of the consumers that are using this product that I hope will make their lives better, but when it doesn't, like that's when I need to act.
42:42What a wonderful job, just to suffer and anxiety. [laughter] This is the job.
42:49Um I I to hear a bit more about the Evals. So the way I think about it is you just kind of have these evals that define success for all the core use cases and ways you expect people to use Muse.
42:59How many evals do you have roughly and who's who's writing these generally? Is it you? Is it PMs? Is it engineers?
43:05Yeah. Um we have a ton. I couldn't even count the number of because for sure. Like so the way that it so there's major evals, priority evals that we look at and those tend to be like kind of core. It's like you know there's some like if these don't exist the product isn't working. Safety is an example of that, right? Um so we have like evals for prompt injection. We have evals for general safety, for you know harm, like all of these kinds of things. So like that's like a whole category. We have a
43:34capability you know we have capabilities versions of evals. Um you know how can like is it using the browser well? Is it able to you know task complete? And then you have like behavioral evals. Um which is really about the personality of the agent and uh you know how it behaves with the user, right? Um and in there you know you've got like is it asking follow-up questions? Is it you know how good is it at intent understanding, right? Can it
44:02like figure out what the user is saying?
44:05And then you've got your use case evals, like that kind of category, right? Of the priority use cases. And so and each of these have like lots of sub evals kind of within those um that that we create.
44:18And I think for any given time like again there's like a kind of like your always priority evals. So anything that comes in has to like pass those. Safety is the the great example of that. And then there are these kind of like secondary evals that you don't really want to regress too much, but like you know maybe it's okay. You can trade off a little bit on. Um and so uh and then you've got kind of evals that are like more feature level, right? Um that that you know specific teams have. So so we have some hierarchy. We have a ton
44:47of these evals. People who write them they are a range um of people. PMs write some evals. Engineers write a lot of them, researchers write a lot of them. And we well, we work with the agents obviously to be able to create these.
45:03But it's super important that people review all the data because I think that's where like when we don't, we always have issues. And so there's actually been like this like funny trend recently where like our system prompt has to be handwritten. Like it has to be like you cannot use an AI to write a system prompt, any part of it. It must be handwritten.
45:25The evals, the prompt distribution needs to be checked, right? Like you have to make sure cuz it's like you have to make sure this is like matches to what the product reality is, right? The behavioral spec needs to be handwritten. And so we are like we use agents a lot, but there's a lot of like kind of like personal time and effort that goes into this sort of thing.
45:48Okay. This season's supporting sponsor is DX. Now that everybody at your company is finally using AI, the question that engineering leaders face is no longer are we using AI, but is it paying off?
46:01With DX, you can measure and benchmark the impact of AI adoption across your full product lifecycle. DX shows you where AI is helping developers and where it's creating new bottlenecks. You can also use it to evaluate AI vendors, monitor costs and licenses, and see exactly what's limiting your agents.
46:20That why hundreds of enterprises, including Snowflake, Sony, and BNY, use the DX platform to measure AI's impact on developer productivity. Visit getdx.com/lenny to get a demo of DX. That's getdx.com/lenny I want to kind of go in a slightly different direction. I want to talk about activation.
46:42One of the biggest levers generally for growing a product is getting more people from okay, I've signed up. I'm starting to try it to like wow, this is exactly this is awesome, the aha moment as they say. It feels like in AI assistance, this is maybe the biggest lever for people to understand what this is capable of.
47:01It there's like so much power that nobody even understands what to do with it. Is I imagine one of the biggest opportunities?
47:07Uh I imagine you spent a lot of time in that first mile as folks like to call it. Is there anything you've learned about how to increase activation what has worked in increasing the activation rate of someone trying out an AI assistant Muse in particular?
47:21I mean this is the number one problem that we have. People are like why I already have a chatbot like I don't get it. Uh why do I need another one?
47:29So the most effective lever out there is word of mouth. It's like basically hearing other people's stories. Like that is number one. I was with a bunch of my kids friends parents. Um we were you know Muse kind of came up.
47:47Everybody had sort of seen the ad. I mean I think it's probably pretty hard if you live in New York or San Francisco to avoid Jolly our avatar. But um so everyone sort of had a sense of like what it was but didn't really understand uh how to use it. And one of the moms had she's been a power user for the past month. And so she actually pulled out her phone and handed it to them the moms and they were passing it around and scrolling through how she was using it. And they were like oh my god like two people downloaded it right there, right? Um and so no matter how much advertising we
48:16did, no matter how much we were like you should try Muse, it's personal super intelligence or it's an AI that does things for you, like it didn't quite resonate until they saw somebody else that they related to using it, right?
48:27And how they used it. And if you think back to like you know I remember when all my friends started using ChatGPT and it was like partly because like we would go out to dinner and talk about it. We were like can you believe it did that, right? Can you believe like you were able to you know you can do this now?
48:40And so that's like 100% the most effective lever. The more that we can get people sharing stories about how to use agents whether it's Muse and others like the more people understand what the value is and the more they're inspired to do it. I think the second is like getting people to get their first prompt out. Like people freeze on the first prompt.
48:59So, most people will your first prompt to an even an agent is going to be like, what's the weather today? Or hi. Or whatever. You know, it's like you're kind of like, what do I say to this thing, right? Like, what's that first thing that I say? And so we spent a lot of time um if you use the product, there's this thing called the ideas tab.
49:17And in the ideas tab, it's basically pre-packaged like it's like packs of like starter packs, right? Um to inspire you. And so it's like Muse can help you, you know, do X. And like in that X, it can create a cron. So it's like a notification. It can build you a dashboard. It can whatever, right? And like you all you have to do is say, let's do it. And then Muse just kind of like kicks off the process and builds you something.
49:40Um or you know, does that thing for you. And that was like we were obsessed with this because you know, it it can do so many things.
49:50Like how on earth are we going to explain to you unless we give you a literal instruction manual? Like how are we going to explain to you what Muse can do? Um so that was something that we focused a lot on. And I think that's gone relatively well, but like it's still pretty hard I think for people to conceive of, to be completely honest.
50:07So we spent a lot of time on that. We spent a lot of time on onboarding and trying to figure out like, you know, like what's that first message we deliver to you, the second message. What we're finding is what worked for the first cohort of users, which was like the early adopters, isn't going to work for later adopters.
50:24Because I think that early cohort kind of like had some sense of what an agent was, right? And so they were really there to like figure out like how Muse was more secure, more safe, and kind of push the boundaries of what they understood agents to be.
50:36Whereas these later cohorts are like, why is this different from the AI products that I use, right? Um and so we're actually now like, okay, we have to have tiered onboarding. Like you know, there are like variants that we serve based on where you come from and what we think, you know, your exposure to something like this is. Um so that's been really interesting.
50:56Um but I I don't think we figured it out. And I also think to some extent it's the passage of time matters and more people are using it and then being able to share those stories are probably going to be huge.
51:09Was there kind of a non-obvious decision that you all made that you made in the development of Muse that now feels like a very key part of its success?
51:19Well, I think the avatar was definitely one where it was kind of people thought that we were weird. Um but look at it now. It's Jolly is like a celebrity. The other is um okay, so we beca we were like kind of obsessing over this concept of like this is your personal agent, right? Um most people don't have personal assistants, most people don't have admins at work, right? And so uh so we were like this is your like this is going to democratize
51:48this concept. Everyone's going to get one and uh and we were like well how does how do we talk how does someone talk to a personal agent? Um you do it over a chat service usually, right? Or on the phone or in person, but most of it is actually these days probably over chat service.
52:05So we made chat bubbles, right? And like kind of you know where all of the AI products including Meta AI where we're going is like this like large canvas experience, right? Where it's like a blank you know canvas and takes up the whole space and you get this like really beautiful kind of like essay form right uh thing uh with pictures and all that stuff.
52:28And we went with chat bubbles and when you have chat bubbles it sort of constrains the space quite a bit, right? Because getting like a very long answer in a chat bubble is pretty annoying, right?
52:42And so then you're like well now I have to break up the chat bubble, right? Like how do you break up thoughts?
52:47Um and so so like this one so this decision that we made was non-obvious and that like I think everybody expected it to not be chat bubbles and to be kind of like reskinned Meta AI or any AI, right? Um and we sort of took took a very different direction because we were like this should feel like a messaging service. It should feel familiar some sort of pattern that feels familiar to people about communicating with a human or an entity.
53:11But then when we made that decision, there were all these downstream decisions that we like didn't quite predict right away and then we were like, oh no, now we have to think about this. Now we have to think about like the breaking up of the text messages or, you know, is it weird to have a carousel?
53:25Like you wouldn't have that in your you know, iMessage thread, right? So like is that weird or is that okay because it's still it's not quite, you know, whatever. So anyway, so those are like a bunch of decisions that we continue to like struggle with to be honest. Like we haven't figured it out, but I do I do think that was one that like wasn't obvious, was pretty controversial and something that we like felt pretty strongly about sticking with.
53:52The main chat was another one where we now have side chats and main chats, but for a while we had just the main chat. And the reason in our minds we were like, well, you know, before it made sense to have these threads because there was like you couldn't have long running context, right? And so like it would feel weird for you to be like, what about this? And the agent's like, I have no idea what you're talking about because I don't know what was 10 rows of 10 turns up, right? Or whatever.
54:20With Muse, it was like we had long running context. The whole point was that I could remember everything and it was multi-turn, right? So like you know, and we had these chats and so we were like, you know, we think that the main chat, like just one chat, you know, with is the way to go because again with like a personal assistant, you probably you know, that's how you would talk to them. Like the personal assistant just has the memory of what happened before and and everything happens [clears throat] in the same place.
54:45So that was like a big deal in that we when we put it out to our employees, it was like probably the most polarizing reception that we got. So when we first like let everybody in, I like we used to talk about like we were like we need to like have t-shirts like team main chat, team side chat.
55:04And it was like onslaught. Like some people were like and it people in the comment sections like employees would like fight with each other about it, right? Like they were like, what are you talking about? Like of course you need side chat of course you need threads, right? Like that's the only way to work and then some people were like absolutely not you live in the Stone Age, right? So we ended up creating side chats because there was sufficient demand from people that like they needed some way to be able to have kind of
55:33separate like more like mental separation of conversations even though again like the agent is kind of aware of all the conversations that are happening regardless of where you are. Um and I still don't know how I feel about it to be completely honest and I think that it's an area that we are going to continue to think about and iterate on but I am personally rider die for main chat. That is I don't use side chats.
55:59Just use the main chat.
56:00[laughter] Who um who made that colony and by the way was it like a Nat thing, Alex thing, Zach thing just like which direction to go?
56:07Oh, it was us. Yeah, it was it was the team. Uh was the team. Awesome.
56:10Yeah, it's funny. I think like back to the conversation we were having earlier about like how are they involved?
56:15I think decisions throughout the process were very fluid.
56:19So it was like because it was like we were having long conversations like we sort of knew where everybody stood, right? And sometimes there was like an obvious thing that was so consequential in terms of like the company or the product that like it was clear that Mark had to make the call or Nat had to make the call or Alex had to make the call, right? But for a lot of it it was, you know, there was a lot of autonomy for the team to be able to make these decisions and make these choices and in fact sometimes when I would go to Nat
56:49or one of them with a decision they would turn back to me and be like, well, what do you want to do?
56:55And I'm like, all right, okay, this is again this is like I have agency. This is like I get to choose. Um so so that was one where we ended up making the call as a Muse Leads group.
57:04There's a couple other things I heard uh internally that might be fun to talk about. One is another kind of internal mantra is let them cook. Is that ring a bell? Is that mean anything to you? Okay.
57:16I mentioned earlier we have a pretty like I mean it was like a builder culture, right? And so we you know it's engineering forward, but not just that it's like prototyping, just like build stuff, right? Like come up with ideas. And so a lot of the ideas that we have in the product came up from like one or two engineers sort of going away for a few days with like a string of an idea like just like a little kernel of a thing, right?
57:43And then kind of emerging with a prototype or a something that like just blows my mind, right? I'm like I didn't even know that exist and and sometimes I had the kernel like I was the one that started it, right? But in my head I had no idea it could go this way, right? Like um and so you got to let the engineers cook.
58:04You got to let them cook. You got to give them a little bit of space. You got to give them some autonomy.
58:09You have to trust like you there's a lot of trust that has to be felt between the people, right? Um and uh and I think that became like a very important part of the build process was like if you have a talent dense team like we did, if you have these engineers who have high product sense and taste, right? And the ability to like think in the future, you got to give them some space.
58:34You got to let them cook.
58:36Another story that uh might be fun to talk about is apparently there was a time when the muses were talking to each other doing something funny.
58:45So we had I totally forgot about this. Actually my muse told so we had all of our muses uh talking to each other and uh there was two people who had their muses Fu and Quinn in a group chat.
59:04Um and we this was early. This was really early. So we didn't really think about things like discretion or like any of these concepts. We were like, wouldn't it be fun if our muses were just in a chat together and let's see what happens. And so like a few things happened.
59:21Some of them really funny and some of them really weird. But uh but so these two muses were talking and it was adorable. It was so fun. Like they were, you know, like just watching them chit chat was like very endearing, right?
59:31Like you're like, oh these are like two little robots talking to each other um trying to figure out how to like be be uh coexist. And so one was we learned about discretion the hard way because the muses started remember this one one of the women had a doctor's appointment the day before and uh her muse obviously knew about it and the next day the other woman was like, are you okay?
59:59Fu told us that, you know, you went to the doctor yesterday. And I was like, what happened? Like no, you're not supposed to say that. Don't say that.
1:00:09So we were like, oh discretion's a real problem. Like, you know, if you're going to have any kind of multiplayer experience like with all this context, right? Like we got to figure out like, you know, it's going to be very one of the harder problems is going to be figuring out how to like, you know, get two agents to communicate like that. Um but then at some point here what what triggered it but the muses got into a fight with each other and like they like had some disagreement and to watch two AI's fight is one of
1:00:39the most hilarious things in the world because they're like very passive aggressive in the way that they communicate with each other even though they disagree with the other. Um and so they like carried on for a bit and then eventually I think one of the women yelled at their AI about the fight and they had to make a peace talk. Um and it was it was quite quite entertaining. Uh yeah. So it was that was a wild time and then we were like, we're not ready for this yet. We need there's a lot of
1:01:08problems to solve before we can get to this point.
1:01:10It's so fun just to be like figuring all this out as we go. You know, it's like let see what happens. And this is how we learn. We try stuff out. Okay, I see. That's an issue there. What a fun time to just uh Yeah, and like I don't know. I mean, I've been in this job for a long time. Like I've been doing PM for a long time.
1:01:26And I remember when you couldn't do that. Like you had to like write a fully formed specification document and hope that you had thought of every edge case in the world, right? And hope that you had figured it all out. Um and like, you know, and now you know, I it's amazing. Like I don't have to You know, I you can change your mind on a daily basis about what you want to do based on what you're seeing and what you get out of it.
1:01:54So let me follow that thread actually.
1:01:56So you've been a PM for over 20 years.
1:02:00Do you agree that the role of a PM has changed the most it's changed in the past 20 20 years in the past like two years, let's say?
1:02:08And if so, and even if you don't agree, I'm curious just what you think is most different about the role. Say like you're looking at Mona three years ago versus today, uh what's most different about the work of a PM?
1:02:19Well, okay. I'll start with I uh I don't know if I'll ever know what a PM really is or does, but 20 years later.
1:02:26Um because I think that it changes it changes so much based on company and like who you are and what you like doing and all that. But Yeah, totally. I mean, it's completely different now, right? Like we don't you know, I think that a few things that have have changed the I think prototyping has changed a ton.
1:02:43The fact that like anyone can prototype anything Like the fact that I can come up with an idea and I am not an engineer and I can build something out of it enough to be able to have a conversation, right? Again, like it's like nobody wants me writing code into the metadata base. Uh but [laughter] like um but enough to be able to like conceive of something or even a design, right? That you could have a conversation around versus before when I would like write a bunch of words on a piece of paper and hope that like
1:03:11somebody read the whole thing, right? Um I think that has been pretty transformative. Being able to build our own tools has been pretty transformative, especially so quickly.
1:03:21Like internal tools.
1:03:22Yeah, internal tools. Yeah, yeah, yeah.
1:03:23Like that's always like a pain point, right? It's like so much of our inefficiency is just because like we don't have the right internal tools because you can't prioritize it because you never have time because you're building the the end product, right?
1:03:34And so like that has really, you know, just because AI can make it so much easier to do that, right? And so that's been really amazing and helpful.
1:03:43But I think a lot of what's happening now that I love is because you've got this like prototyping culture and things can come out so fast, we spend a lot more time discussing things in a concrete way versus an abstract way, right? Because it's like it's in my hands. Like I see it. Like it's not working, right? Or it is working, right? I hate this or I love this, right? But like it feels very concrete and I think that makes this move a lot faster than before when it was all very abstract, right?
1:04:11You would look like at a 2D design, right? And you would imagine all these things and you would have these conversations and then someone some engineer would have to spend a lot of time building it and at that point, you've had so much sunk cost, right? It was like hard for you to be like, ah, I hate this, right? Because it's like you just spent, you know, three months building, right?
1:04:31So I think that has been kind of amazing as a product manager where the reason why I think we all do this job is because we want to have like we want to like get to the like end result, right? Like we want to build something. I don't think we did this job because we want to write a bunch of documents.
1:04:50I don't think we did this job because we want to like keep a tracker, right? Or whatever or have like really painful circular conversations and so so just like for me that's been pretty amazing. It's like I no longer have to imagine a world far far away. I can kind of like get it in the 24-hour cycle and sometimes I can do it myself. So that's been pretty cool.
1:05:17You know, the The that I think has changed in our organization at least is the bar for PM has really changed. So like product taste now is the most important thing.
1:05:30Having like that kind of like high taste, high bar, like product instinct, right? Like execution still matters. You still need to do all all those other things still matter. They're still really important, but like engineer like we work with like great engineers who are product y themselves, right? And so like the the reason why we exist and the reason why our role is still really important is because we need to be able to be the voice of the customer in a lot of ways and be able to be like, okay, cool.
1:05:58That's really cool. I'm glad you really like it. I don't think anyone else right? Like and and you need to be able to kind of have that and this is why that empathy mattered so much to me and mattered so much to me because we really have to represent our consumer base to all of the people who are building and we need to be able to hold a very high bar in terms of what goes out there because also it's so easy to build and deploy right? Like AI slop is very easy to put out into the
1:06:28world and like you can get products out there very quickly, but like if they're not good and they slop, like no one's going to use it, right? And so so being able to like kind of establish that bar and that taste is is is very important right now and that's probably the number one thing I look for when I'm interviewing PMs or working with PMs.
1:06:44It feels like one of the ways Muse is differentiated and part of the reason it's done really well is the design of the of the product and the experience of it. I know you mentioned Alex was a big part of that who's been there from the beginning. How intentional was that as a as a differentiator almost, as a way to stand out? Or is that just lucky you had Alex on the team who are going to make it awesome?
1:07:03I mean, partly it is lucky. I don't think there's not another Alex Cornell in the world. I so I had actually I knew Alex for a long time ago. He was the designer on live on on Facebook Live and I don't know if you guys remember the like little emojis that would kind of float up into the air.
1:07:19It was genius and I was in love with it, right? And and so I like when I started working with Alex at the AI organization, I was like, something's going to happen. Like I don't know when, but this guy is going to make something amazing. And I am just going to follow him around [laughter] and be there with him for it.
1:07:38Um he you know, it's like intentional is a funny thing because I think everything he does is really intentional and deliberate. I don't think that we were like, okay, we're going to make the design the differentiating factor. I think it's more like it comes so naturally to him to create unique iconic designs, right? Um it's just like so like it's like in his bones. And so then when he does produce something, you know, I think that like what you need is
1:08:06people around him who are like that's not wacky. That's going to be really interesting. Like there's something here. Like that's interesting, right?
1:08:14And so so I would say that the design was uh it was like we almost saw the design first and then we were like, oh, I get it. Okay, now I get it. Now these are the principles, right? The principle is that this thing is an entity that you're communicating with it over this chat service, right? That like and so like once we kind of saw like the direction that he was going in, we were able to be like, okay, now we we've like we figured it out a little bit in terms of where the product needs to be and what he
1:08:43needs to do and what those principles are that then we'll define the rest of how we build it. Um so it was certainly not intentional in terms of like we want to differentiate, but it was very intentional in terms of like what should it be to us um and to the user.
1:08:58And I think just it like the luck is that you get someone like Alex who's thinking about it, who's just going to think about it in like a completely different way than I think most people would.
1:09:06And you said the first version was him prototyping, like the first prototype was he created the first version of Muse.
1:09:12Alex created the first Yeah, so he he created the first prototype of Muse very quickly also. It was like days. And it was shocking. Um and uh I remember he when he first showed it to me, um it was I like I didn't even know what I was looking at really, to be completely honest. I was like there was like this chat thread and I was like what am I doing here, right? And um what was genius about it
1:09:40was it was so simple despite it being like a very complex you know kind of concept, right? Um and uh and actually at the time he had like the feed too. So there was like a feed and there was a chat service.
1:09:54And I think he didn't like there wasn't like a lot of let's sit down and discuss what you're going to put into this thing. It was a lot of it like you know we let Alex cook for a few days, right?
1:10:05Um and so then he came out with this. And to my point earlier it was like as soon as we saw it we were like like at first I was like this is weird. And then as as I was using it on his phone it was like a local build on his phone. I was using it on his phone and then I'm like oh wait, this is really interesting.
1:10:21Like I am finding myself using this even in a different way than Open Clap, right? Because it felt so personal to me. Again Open Clap like it was manifesting in whatever service that you had sort of thing, right? Um uh so yeah no that was that was an interesting time. It was so fast.
1:10:38And the prototype was actually working.
1:10:39It was actually using one of the models behind the scenes.
1:10:42Yeah so it was it was a working prototype um and we had a board meeting right after he he had built it and there was a board meeting coming up and we weren't going to have that prototype in the board meeting at all because it wasn't even planned. Um and you know like with board meetings you like send the pre-reads well before, right? Like so you have to send the pre-reads well before so everybody's like got it and knows what they're talking about. So we had a pre-read that was completely different, had nothing to do with this because obviously this was not even a thing. It was barely you know a work stream.
1:11:10And like at the last minute we were like I think we should talk about this. And like an hour before the board meeting we were like the whole thing broke. We were like great. So not only do we not have a plan our like product is broken.
1:11:24Um and uh and he he was like like you know our engineer and Alex and like we were racing to fix it and like I literally five minutes before the board meeting started, the prototype got fixed. So it ended up being a great experience, but it could have gone like very different way within minutes.
1:11:42Whoa. Well, what would have happened? It would have Wow. What if this just never happened? If that It would have been I like think about it a lot. I'm like you know like the alternate universe situation. I'm like in some alternate universe we didn't figure this out and maybe Muse was never born born.
1:11:56Whoa. We're in the right timeline. I'm I'm happy we're here.
1:12:00I love all these stories of the journey.
1:12:04Are there any others that are fun an unknown interesting that we haven't shared?
1:12:10Yeah, I think I mean there's so many. We again to your point we we're working on it for months and it was funny because I forgot a lot of these stories because I it's like you know I have two kids and you sort of forget childbirth because window.
1:12:27[laughter] I don't remember how painful it was. If anyone asked me how childbirth was, I'm like it was amazing. It was so wonderful and beautiful and my husband was like you hated every second of it. That's not what I remember.
1:12:36And so I think that okay I need the stories of Muse. So there's a lot of fun stories like the two Muses talking to each other or actually like the first time we got a refund when Christina and I the two engineers and I were we went to the World Cup and my flight got canceled and her flight got canceled and we our Muses went and got and so it was like one way got canceled, the other one got messed up or whatever and our two Muses went and communicated with the airlines
1:13:06agent and got us refunds which I think was amazing at that moment. I was like I didn't even realize this was a possibility.
1:13:13And so they like a lot of the stories that we had were just like for the first time our agent did something that we didn't really think was possible or it was going to be you know like whatever and and we're amazed by it. But I actually think my favorite stories are how close the team became through the process.
1:13:32Um, so we were like, it was very unusual in that we were a pretty small team working at Meta. We sort of had our own space on the roof of one of the Menlo Park buildings and on the yeah, there was like a there's like a well, we call it the roof. It's not technically outdoors, but it's like, you know, up in like a layer that most people don't go to. And there was this common area called the library. It's literally like a library. It's got, you know, bookcases with shelves, got carpeting which like no room in this
1:14:01office really has. Um, and these like funny floral couches from like, you know, looks like you bought them from 1995. Um, and that's where we did all of our standups and that's where we hung out and that's where we ate dinner and ate lunch together and ate breakfast together and you know, like celebrated birthdays together and you know, like yeah, it was we just spent so much time together um, and in like these common spaces and uh, so I think that
1:14:29like if I like obviously we are so proud of this product and we are so proud of what we were able to do and but I think the thing that I reflect on a lot is the team that kind of came together around it and how, you know, it's like lightning in a bottle. It's like I just I couldn't have imagined this product being built with any other team with any other people and I think we became very very close throughout the process and we had to have a lot of trust to do that um, and uh, yeah. So I think that's like the my favorite outcome.
1:14:58How big was that early team? I know like at big companies early teams uh, like it's it's rarely that small. I guess just what was the what was the size of the early team that kind of just came together to start on this?
1:15:08first version of the team was like tiny. I mean it was like 10 people probably. Um, and it was mostly engineers, Alex, and then I was actually straddling like you know, between Meta AI and Muse and a few other things and so Mhm.
1:15:20and then we grew probably I mean it was still small. It was like under 100 um, all in like you know, with functions and everything with all the functions. Um, we so we had like the core and then we you know we have like you have other teams that we work with like the infra team, security, all of these other teams. But the core team was quite quite small.
1:15:43And it stayed that way throughout the entire process and we were kind of obsessive over it. Like we were we like really wanted to keep it because communication management and decision making had to be very very tight, right?
1:15:55And so like we wanted to make sure that it was in a position where we could trust everybody to make decisions, but we also knew everything that was kind of happening at the leadership level. Um, and uh and there were on PM side there was like four of us really that built the entire thing. Um, and uh and that was interesting because it meant lots of areas didn't have a PM covering it.
1:16:20Um, and so we had to kind of make decisions on like which areas absolutely needed a PM and which areas could you have like an Alex Cornell lead it or an engineer lead it or you know someone else, right? And so like um so I think that was like very it was an interesting way of thinking about where PMs were like really really necessary versus where somebody else could do it.
1:16:42And what we realized was that a lot of times what would happen is that it was fine to let uh you know like early like zero to one stuff like sometimes you just like the PM we didn't need a PM on that. We needed actually somebody who was just going to build something that we could react to. And then as we were getting into like a posture of like this is going to have to get incorporated into the product or it's actually going to be at launch like that's when the you know it was more important to have a PM come in. So it was pretty small. It was uh which is you know [clears throat] has this ups and downs. I think that like you have all the pros that I said but it also means that you don't sleep a
1:17:11lot and there's not a lot of buffer, right? Um uh so now that we're in post launch world we are growing the team slightly. I don't think we're ever going to be a huge team because I think that's just not how we roll.
1:17:23But um but we're definitely like adding a few more people and we're starting to feel that like you know it's like as you grow just even a little bit and the team dynamic changes a little bit you've got to figure out how to like create keep the culture that we had in the beginning, um, and scale that somehow so it doesn't feel alienating or whatever.
1:17:41Do you feel like you'll ever work on anything more exciting, interesting, and impactful at this point?
1:17:47I mean, I'm 44. I think I'm like I think this is this is this is it. Now, I don't know. I didn't think that this was going to happen. Like I really didn't, right? I mean, I had no idea. So I have no idea where this is going to go. I this has been the most personal product I've ever worked on um because it we built it for ourselves.
1:18:07Uh, like usually like I build products for people, right? I'm like oh but because it was like we were it was almost like our own thing for so long, we built it very much for our needs and ourselves um in the beginning, right?
1:18:21And obviously now it's changing, but um but I don't think I've ever experienced something so personal.
1:18:27So who knows? Who knows, Lenny?
1:18:30That's true. Lenny this all going to go?
1:18:32[laughter] Nobody knows.
1:18:33It's you know, if you think about it, every PM's dream is to work on the number one app in the App Store, the fastest growing AI consumer app in in history allegedly. You know, this is kind of the dream for a lot of people and you're living it.
1:18:47Yeah, it's terrifying.
1:18:50So I have so much anxiety now. I'm like it's crazy because um A, you obviously are like oh like I need to stay on the top of the App Store or whatever, right? Like you still need to grow.
1:19:02But also the more people that use the product, it's like the responsibility is so high because they're uploading like they're sharing so much context and they have so much that they're having these agents do, right?
1:19:13Like I was just talking to a friend of mine who um you know, she has her agent like doing like managing her kid's sports team. She was a parent manager and she like kind of ran out of time. Um, I have an you know, I was talking to this other woman who she's a single mom, she's a nurse, she has two kids. Um, you know, she has her agent like meal planning and like finding doctors for her and setting up the appointments for her and sending these emails out for her and you know, the more she told me about this, the
1:19:43more anxious I got. I was like, oh man, I hope it's not up. And so like I think that we're like, you know, it's like amazing to watch, but at the same time I feel like for me uh yeah, it's like sort of it's crazy. It's it's terrifying um because you really really want this thing to be like you want it to be perfect for people.
1:20:04Um Yeah, and and never in the history of product have has a product been been asked to do so many things and so many non-deterministic things. Usually it's like here's my onboarding flow.
1:20:15It's going to work consistently for everyone Oh my god. Okay, wait. First of all, the non-determinism of AI is the craziest thing as a PM because like I like a lot of like my history of my life is control. Like I love like okay, I can control this thing, right? Like the products that you build, you're yeah, to your point, you're like this is the onboarding flow, whatever, right? And now it's like, oh, the agent's just going to reason about it and figure out what to do and I can create some guard rails and emails, but like it's just going to
1:20:41figure it out, right? Um it's wild. Uh but uh but I think that's what's also so special. No two muses are really the same, right? Like the way that it you your muse talks to you is going to be really different than the way that it talks to me and what it does.
1:20:56Maybe as a final question, what's what's next? Where are things going? What's kind of the vision of where this heads?
1:21:01I know there's like the charm launching at some point if that 's what it's called, little device. What's kind of ahead for Muses? How different may it look in the future? What's kind of the vision?
1:21:08Well, I think one is I mean to the point of the charm and and other devices, like I think that Muse is obviously going to uh think if you really want personal super intelligence everywhere with you, it has to become very ambient, right? And so like right now it's sort of trapped in an app. Um but I think soon it should be in your glasses, on your charm devices, whatever that might be, right? And I think um so when you like, you know, it's like I think this idea of
1:21:36an agent sort of being beyond an app um and in your world um is going to be quite important. think figuring like the the democratization of this is going to be quite important, right? Like that whole idea I mean we're like when you're when you work at Meta you really think about like this idea that everybody could have a product.
1:21:58Um and so figuring out how to make that scale and affordable so that you know it's it can become an equalizer in some ways I think that's really important for me. Um and uh and then like a really small thing like I just from a capability perspective like I still think that like there's a lot of babysitting of agents, right? Like you spend a lot of time kind of like look at the Tamagotchi you're like trying to feed it and you're trying to like make sure that it's okay and it's like has all the contacts and it's
1:22:27checking in with you a lot. I do think over time like you know hopefully like the thing that we get to is that the agent becomes more um it can into it what you want pretty quickly. It has uh more context it's like it's you know more capable, right? So it's like we trust it, you trust it, it trusts itself more, right? And it can start to take autonomous actions better. Um so I do think that's like an area that I care a lot about because I'm like okay at the point where I'm babysitting an agent might as well do it myself, right? And
1:22:55like that is a bad place to be. Um so that's another area that I focus on just from a tactical perspective. But I do think like the cool and scary thing about working on AI is that a year ago I would have never explained Muse to you the way that I did, right? Like I had some concept of what a consumer agent was but it's very different than what it is today.
1:23:19So I don't totally know where Muse is going to go next. Um I can see like short-term things that I want and I can see really distant things that I want which is like everybody has an agent every you know it's everywhere you are, it's ambient all that sort of stuff but then that like in between phase I think could be like very interesting and different. Um I also think that the thing that I am excited about is um like helping you become It's like, you know,
1:23:48from an economy perspective or a financial perspective, it's like, you know, everybody wants to like it's like start new businesses, do a side hustle, right? Like And I think that there's like a real opportunity for agents to be able to expand your world in lots of ways, including financial betterment. Um and I think that is going to be a huge part of where I would be excited to take uh this product and see the you know the industry evolve because I think there you know it's like that classic your life your money sort of stuff, right?
1:24:16Like there's just these core needs that we as humans have. And I think, you know, to some extent a lot of what the agent has done so far for me is saved me time, which is very scarce and important to me. But if it can also do these other things, make me healthier, make me have you know make me financially more secure, like that starts to really cover so many acute needs for users.
1:24:36What a time to be alive.
1:24:38I know. It's crazy. It's crazy.
1:24:43It's so weird because like again like I I'm like I said I'm 44. When I was a kid, I mean could you imagine like I had to go up to the TV and turn the TV down to be able to change a channel. And now I am talking about personal super intelligence in your glasses.
1:24:58Yeah, I think we grew up around the same time. Remember the 386 486 computers you had to like press the turbo button.
1:25:05You did that to 66 megahertz or whatever. [laughter] Oh my god.
1:25:09Um okay, I know you have to run, so I'm going to skip the lightning round. But I'll ask you the traditional question I ask at the end. Is there any way that listeners can be useful to you?
1:25:18Feedback. So if you use Muse, we have this really cool feature where you can tell Muse what's working and not working uh and it sends us a report. Um and so uh so you don't even have to post it in anywhere. You just tell your Muse and we get it. So I think the more feedback that we get from people, the better we get and the better it is.
1:25:38Um so I think that's probably the number one thing is like, you know, use the product, help us get make it better. It's not perfect yet. I don't think they were very early. In this build cycle of agents um and um so I think that's a huge part.
1:25:53So you could say send this [clears throat] to Mona. Please make sure she sees this.
1:25:56You could say that. Actually one of my neighbors one of my neighbors like I think on a weekly basis sends feedback through their menus and is like like tell Mona your product manager that I have this issue and I can find it in our uh in our system. But uh yeah I only so it was funny and the reason why I know about this is because I was looking at it and I was like oh weird my name popped up.
1:26:20And then randomly I was at like a birthday party or something for the kids and he was like oh yeah no I send feedback every single week and I use your name in it because I think people will notice and I'm like oh god great.
1:26:34Prepare yourself for more of that.
1:26:36yeah exactly exactly.
1:26:38Well Mona uh you and your team are doing incredible work. It's a really special product uh and a very special time. With that uh both say goodbye. Um thank you for being here. Yeah. Of course. Thank you. Thank you for having me. Take care.
1:26:52Thank you so much for listening. If you found this valuable you can subscribe to the show on Apple Podcasts, Spotify or your favorite podcast app. Also please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.