0:00people are going to copy you, right? And so the your product doesn't stop at your features. So you have to be thinking about the entire experience and you have to also be thinking about how are we going to be distinguished with our data.
0:14But who are the AI companies who are getting the the most funding right now?
0:17It's actually all the ones who are data focused.
0:24Hey everyone, welcome back to the product experience. I'm Randy Silver and the thing that's come up time and time again over the past week is people talking about moes. Not moes around castles, not dragons, not know hordes of gold, although corporate hordes of gold do come up in this. So, uh, we brought in someone who's an expert on the subject, Rob Ferguson. And Rob, do you mind doing a quick introduction to yourself? Where are you now? What are you doing? And how did you get into this
0:53game in the first place?
0:55So, I'm Rob Ferguson. and I'm the vice president of technology and strategy at Fireworks. I used to be the uh global head of AI for AWS and I was the CTO for Microsoft for startups as well. I was a a startup CTO and you know early at many startups which have had three exits and so I've kind of been in the technology game for a while now and definitely have done a ton of startup coaching as well
1:22as helping people kind of understand how does product strategy and you know your go to market actually affect your moat going into this new age of AI. So let let's start at the very basic of this before we go into the the complicated interesting stuff of what's changing in the world. But how do you actually define a mode and is it is it purely one thing or are there multiple
1:52approaches to building a mode around your business?
1:55Well, so you know, a lot of times I think people worry about modes a little bit more than they should because I ultimately it comes down to competition and competition is usually a good thing.
2:07I remember when I was the CTO of place called Automatic, uh we were building an adapter we would plug into your car and it was a first of its kind. We were really excited, but of course immediately bunch of other people saw what we were doing. We saw all of these little carbon copies on it and we would see these carbon copies. They would have our exact icons in their app.
2:31Um, same thing happened, right? When I worked for Ario, which was a music service, um, we would see our exact same icon, but it would just be rotated a little bit.
2:44And so, I think that people are always thinking, you know, how how do I deal with competition? How do I make sure that I built something that is a durable business and that it can has that sort of sustained value? So, emote is competition and value and understanding how to keep building and make sure that it doesn't erode over time if somebody doesn't come in and take your lunch.
3:07So, there's there's lots of ways you can build a mode, right? So, you can be about reach and distribution. It can be about customer service. It can be about uh lock in in any number of ways. Is there something that you think is the most defensible approach or something that you, as you said, you've been an adviser to a bunch of startups? You've done this a number of times yourself.
3:26What's the the primary thing that you advise people on when they're thinking about what their mode is and how to approach it?
3:33You know, I I do actually think a lot about customer service. I do think a lot about understanding who your customers are and having a direct relationship with those customers because there's always that that energy that it takes to go out and really do things right, really be present with the people that you're working with.
3:56That is not a small burden for competition to come in. Of course, they're going to try to go in and you get a lot of spray and prey kind of competition. you know, people who are just like, "Well, I wonder if there's business here. I'm going to put a feeler out." A lot of times I see folks who uh relentlessly worry about competition.
4:15And the good news is that competition means that other like people think this is interesting. People think it's valuable. I like I'm always welcoming good competitions because it just it's gonna help me be differentiated out in marketplace because I'm gonna I'm just going to go out there and do a better job than anybody else who's there because I'm going to care more. I'm going to put the investment into it. I'm going to really think it through it. But these days, uh AI does make things a little bit more complicated.
4:46you know, uh the it used to be there was a barrier entry to like put up a SAS site and now like you know the other day I was looking at a Korean barbecue site that looked exactly like your typical SAS site, right? You know, somebody just went into, I don't know, Lovable or Copilot or something and said, "Give me a professionallook website." And here it was looking exactly like a traditional SAS website. Probably took them five minutes. And so I do think that the the
5:15story of how you go to market has evolved a little bit. And I do think the story of what makes you defensible has evolved a little bit. But at the end of the day, it is definitely going to be your ability to understand your customer and your ability to go and have a direct line of communication with that customer and know them better than anybody else.
5:36So it's not enough to just throw something over the border. that works when you've got something that's kind of established already. But we're seeing things change. So, you know, the the most recent example uh that came up was Jev uh launched what about two weeks ago now. And within days, I mean that was something that took about two years I think of of hard scrabble development and a lot of money, a lot of time, a lot of effort. And within days, maybe a
6:05week, there were competitors that were at 95 to 97% parody on it. In a in a world where features used to be a bit of a moat, they would take time, you know, months to years to for people to to copy and that these are things are now showing up in days.
6:26What's changing? What, you know, how can you be comfortable distinguishing yourself? And yeah, I I don't even know where to go with this. I mean, features used to be something you executed really well. You're really proud of it. And now it's just not something you can distinguish yourself on, is it?
6:43You know, I I'll push back a little bit and I will use Jeff as the example. So, it's funny that you mentioned Jeff because our team a year ago published a blog and you could go find it and it's basically like I think the title something like how you could train a classifier to do this for $11 and not that many people really engaged with that blog at that time even though like our research team had spent like a lot of time thinking about it. The
7:13barrier to entry there wasn't just building the classifier which is effectively what JV is. The team there has done such a phenomenal job. You go and you can look. They launched with all of these cookbooks. They launched with all this information on how and where to use it. A strong community effort really kind of going through and taking an idea that I'd say was in the periphery and now it's something that like you're you're in the discourse around and
7:40that's hard. R rising above the noise in the AI era is a huge feat. It's incredibly difficult. So all these people who are going off and they're saying, "Oh, we've got to clone this thing." Do you expect that they're going to update it the next time that it's not kind of newsworthy for them? Or are you going to go back to Java and see what they're doing? Right. It's it it's rough. It's a little brutal out there that getting above the noise is much more difficult than it used to be because there's noise everywhere. One of
8:09the startups that I advised, they said that, you know, the previous cold open rate on emails was 4%. And do you know what the cold open rate on emails is now?
8:23Oh god, it's going to be fractions of a percent. I'm guessing it it's just zero. Like people just don't open these cold emails anymore because it's all this AI slump on the other side of it. And so what used to be a hard but useful channel 4%. It's now not a channel. Right? So the ability to move above the noise is hard. The GE team I think did a phenomenal job. It is so complete. They did such a good job of
8:52taking their time to market, understanding who their audience was, building a point of view, delivering it across many different resources, engaging with the people to not only come above the noise but also give them space to think about building other products on top of it.
9:06I think you know it's like it's it is so difficult and the fact that it came to you so quickly even though there's like that like it existed other people were trying these things didn't make it there but they understood who their audience was and they built something custom that made a difference for the people like me who were always trying to think how can I do more and they made it like kind of personal okay so it's an interesting one I I come out of music journalism and my past and
9:36the the way I tend to think about this is the means of distribution. You know, the people used to literally own shelf space in record stores and those were the major labels and that's why Exactly. And that's why bands would sign with them. The same thing with grocery stores in anything with physical retail. But now there's infinite shelf space and anyone can theoretically break through.
10:00But you still it takes a lot of money and it takes a lot of effort to launch and to market yourself. Well, so that's what Jev has done. And then a bunch of competitors came along and were uh following in their wake and and doing drafting off of them and got, you know, some real traction very quickly. And that what I'm hearing you say is it's that initial launch is important. that drafting is very opportunistic, but then it's the long-term following up
10:29sustainability that continuing to innovate, to expand, to deliver. That's the real secret.
10:37Yeah, I think so. Um, and I think it's it's you can't stay still, of course.
10:43And you know th this is the hard part about this era for me and I you know I could just speak to it very personally which is you know at fireworks we are helping people not only build custom intelligence but we're also always curating the latest and greatest in open AI and what that means is that we have to stay on top of so many launches we have to go and under we have to develop a point of view of like you know is our audience is going
11:13to care about this? Are they going to use it? Can we optimize it and make it work better for them? You know, we're always trying to really push the frontier for people, but that means staying on top of one of the most active, busy markets you can imagine.
11:28And so, operationally, it can just simply be exhausting to try to figure out with everything going on what what should break through the noise. And I think curation, you know, this is always a problem, but now it's now it's in my industry. It used to be it just in like music and art and things and you know you would have how many people make art?
11:49You know, everyone in my family makes like a little art, but I guarantee you a lot of that art is not stuff that needs to be making it to your magazines. No offense to my sister, but like it's it's very difficult to rise above there and to be like you know it relationships start to matter, right?
12:10the understanding what connects with people. There's a commercial element.
12:14You don't want art to be commercial potentially, but of course it comes into the fray. Like, you know, people think, could I put this in my living room? You know, are do you have a point of view?
12:24And all of those things that maybe felt a little bit less tangible, like it just felt like all you have to do is be technically good. It's not enough. You can't just be technically good anymore.
12:35You actually have to be a taste maker and you need to be able to bring this point of view which means building your own intelligence having the point of view understanding who your audience is and connecting with them. That is a lot harder for technologists who sometimes want to be introverts and just solve difficult problems.
12:57Let's I I really want to talk to you about where this is all going as well. Um, so the cost of compute is something that comes up a lot. You know, there's the CEO's point of view of or the last couple years, let's get rid of lots of people. We can just throw AI at this.
13:15It'll be much cheaper. But then the cost of tokens comes in and that can be quite expensive. Uh, how well you use the tokens can be quite expensive. the open weight models are catching up to frontier models pretty quickly. There's the question of can you do this on your own tin or your own data centers versus doing it buying it from somebody else.
13:37Where do you see this all going? What do you what where do you think the trends are going? I'll ask you to, you know, look into your crystal ball over the next couple years. Where do you see this going?
13:48So fireworks, we're dealing with the biggest of the big bottles. You're not going to put this on your Mac Mini at home. So, we take super clusters which on their own are hard to procure. You know, I think a lot of people when they think, oh, a GPU is a GPU. No, like we need physically proximity GPUs because the information that's going through is so dense, right? And we buy on 20 different clouds where we go and we find the best deals in market that we can.
14:17And kind of like Costco, we pass that value along to the people that we're working with because these GPU costs, I mean, we're talking real dollars. You know, as a back of the napkin estimate, say like 10,000 GPUs represents a billion dollars. And then, you know, these models, you're looking at, you know, on the small side, you might be like 80 GPUs or more. And then you want to have uh you want to have some replication of this because you want to have service
14:47quality on the other side. So it's not enough to just have the GPUs but you have to keep them busy because idle time on GPU drives up cost for everybody. So which you don't see lots of people who want to go ahead and just be like hosting their own GPU infrastructures and models direct. We we provide a serverless infrastructure so that just like OpenAI and Anthropic do as well.
15:10That's that's the standard at the very high end of AI. So those GPUs stay busy because otherwise those costs get passed down to other people. So we're out there really being very innovative in terms of performance and keeping you know the the intelligence that people want locked and loaded in a way that they can use it wherever they need to. Whether it be oh I needed to be US only or I needed to be able to host some custom weights. All of these things are things that fire
15:39provides and it's a tremendous amount of technology, physical and human resources to put all that together. So from our perspective at the the high end, the very large reasoning models that do the most work where you want to be really at the frontier, you need to work with a partner who can go and manage all those things because otherwise uh the GPU hosting cost you're going to be trounced, right? you're going to experience what it's like to have idle
16:07time or what it's like if you had to figure out moving these large models in between resources because you're keeping 100 plus GPUs occupied that whole time which is I don't care if you're Microsoft I don't care if you're AWS that is a real cost and so being operationally excellent and hammering that is the is like the bottom layer to be able to do this In other parts of AI, we are going to see more and more. We're
16:36going to see multiple of models. You're going to see different kinds of intelligence delivery. You're going to have some local models. You're going to have some models that are made to be really quick like an audio. You're going to have this whole distribution of intelligence. We're all going to learn that graph kind of together over the next few years. But it no one wants to have to go through and maintain this completely on their own. need really excellent people to go through and take some of the burden out of the
17:04intelligence. That's the the first layer on like your journey to AI is kind of figuring out kind of who your partners are and who like you know what kind of intelligence you need to bring to the table and all of that is going to come back to your data and your customers. So if you can tell me, you know, a little bit about the kind of data you have and a little bit about who your customer is and we can figure out how to map that intelligence really, really well. But I think a lot of people are still kind of
17:33early in their adoption where they're what I would call model dependent.
17:37They're just hoping that the next model that comes out just happens to work for them. And so usually I tell people fireworks is your first step on your journey to model independence, right?
17:48That's when you start to know how and why do you want to use intelligence?
17:51What are the trade-offs that are there?
17:52And how do you get the best value for yourself and your customers?
17:56Okay. I definitely want to talk to you about using models well because I see lots of companies not using AI or or models very well. But it it sounds like what Fireworks does. Uh the historical analog I would use is during the gold rush there were the people who were the prospectors who were going out for the gold and then there was Levis and other companies like that that were selling the tools to the to the people going out and prospecting. Is that what an a reasonable analog for fireworks? You're
18:25selling the infrastructure to enable people to to dig for gold.
18:30Yeah, it's a funny infrastructure. I usually say we're your favorite AI's company's favorite AI company. So whether you're like cursor or notion or you know Heidi Health or any of these really phenomenal AI natives, I I figure like the way that they became an AI native is that they found fireworks because before you're this AI dependent, right? You're just like hoping the big labs are going to provide what you have.
18:58But how are you going to find your mode on the other side? Uh if you're at the point where you're spending, I don't know, $15,000 a month on inference, you would have already seen a return if you start thinking about building and operating your own intelligence. But AI labs kind of obiscate that from you.
19:19They don't really give you a lot of options, right? They're like, okay, you can choose the expensive model and the slightly less expensive model. Those are kind of your options there. But we help people customize intelligence for their application. So for example, cursor, which is an incredibly popular AI coding platform, you know, those models didn't need to be able to uh or mouse uh tell you all about your health. So what they were able to do is they were able to
19:47distill a model that didn't really focus on things that were not AI coding and as a result able to offer a way better cost per intelligence for their customer at a way better speed. And so the way I think about it is, you know, I all the startups who come to me, they're like, "Hey, Rob, I want to start a startup. I want to figure out AI. What should I do?" I'm like, "Well, there's so many subsidies in the market right now. Go use up all
20:14your anthropic, all your open AI, AWS, whatever is out there. um figure out kind of how you're thinking about your intelligence and then when you're ready to actually like build a business on top of that, come to me because we're going to figure out how to take that intelligence and we're going to make it work for your cost and scale and for your customers in a way that instead of having, you know, uh I would call it exponential AI costs, you have costs
20:42that make sense for your business.
20:44So we we kind of went through something similar with the launch of cloud in that people were choosing stuff and then they got locked into a specific provider whether it was AWS or or or anybody else. Uh and it was very expensive to switch but switching costs are next to nothing now. And what you're talking about with with Fireworks, you're helping people pick the the right model for the right context at the right time.
21:13You'd think that to a certain degree this is a feature anthropic or chat.
21:19They could be offering this amongst their own models, but I'm not seeing them do that or at least not very well right now. Wh why not?
21:28You know, at one point Open AI did try to allow you to customize models. Um yes, didn't work for that well as far as I could tell.
21:37Yeah. you know, I don't think that they even you map your business to where where the opportunity is, which makes sense. Um, I also just think that it's harder when you're dealing with closed links.
21:51Fireworks, we really take this time to let people use the very best in open intelligence so that they can inspect everything. If you want to train a custom model on fireworks, you can take that and then use it somewhere else. It just happens that we're the best in training and the best in inference all on one platform. Most people don't realize that most AI training, building your own models requires also excellent inference because uh it's part of the training process. you actually want to
22:19go through and use your examples where you are inferencing across the best open models and having it all in one platform means that you get really like you need your inference costs to be reasonable but also they need to be lightning fast and we're performance experts. So we make sure and we figure out all the ways to optimize and some of the internals of that are incredibly complicated. We spend a lot of time focusing on things
22:47like numeric. It turns out that if you have a tiny tiny fraction of a decimal point that's slightly off, for example, because these these models have multiple layers, you went to the wrong layer and it got a tiny fraction of a decimal point off. That will show up in the final result. You'll actually see it on the other side. And so we're the team behind PyTorch. we which is the framework that I don't know probably 95%
23:16or more of these models are trained on and so we understand the very very internals of the whole system and so we've long been considered the best option for AI performance but then we also added the ability for customers to then go and specialize their own intelligence with it the end result is that it's the lock in is that it works really super Well,
23:42and like I know that that's a funny piece of lock in, but we also provide access to researchers with fireworks labs. We have a forward deploy team.
23:51We'll go in and we'll help you figure it out where we just really focus on excellence throughout the system. It's not that the other AI labs couldn't do this. It's just that they benefit for things being closed. They benefit if every time you have to upgrade to the next model. So they don't have to show their receipts for everything. They kind of want it to be a little secret because that's their opportunity. We're the opposite. Everything's out in the open.
24:17You have the optionality to move between things. But, you know, even Kim Kardashian shops at Costco, like it's really good to be able to have like, you know, really excellent olive oil that you can go and get where you're like, you know, it's going to be really high quality. You kind of know it's not going to be every single thing that's out there, but what's there is great. And then since we offer that customization, it makes all the difference.
24:41Quick one for the product folks listening. AI can draft the PRD, prototype the idea, and help you build almost anything. But knowing what to build, that's the hard part. Jira product discovery helps product teams bring customer insights and ideas together, prioritize what's worth building as a team, and create living road maps that connect directly to delivery in Jira. So when more ideas are possible than ever, your team can focus on the ones that matter most. Try Jira
25:11product discovery for free today at atassian.com/mtp. So I'm seeing lots of companies and lots of developers go to it down one avenue.
25:24They're they because the tools are easy to use because LLMs are the thing that everyone's focusing on and it's got a great interface and almost an addictive interface. That's what people are using for everything. But there's a lot of other things out there that are under the hood of AI, ML and Rag and a thousand other things that are probably better for lots of these things than using LLMs.
25:49What do you What are you seeing the best companies do? What do you are you seeing people trapped in this loop of just using a chat interface or are you seeing people actually use this intelligently?
26:01So this state-of-the-art right now the the best model in the world it's not table it's not Astra right it is always a mixture of models you can always if you want to talk about the the very frontier of intelligence the search area of intelligence where you're trying to do brand new and interesting things well it turns out that the models underneath they have affinities you know some of them are better at JavaScript some of them better at Python some of them were you know they they have different pieces
26:29of intelligence that's And we can reliably reproduce that if we do what's called oracle modeling. You go through you do a set of tasks and then afterwards you run every pure mutation of combinations. You literally go and just check what if we used every single thing. What would be best? The answer is always a mixture of models. And so the very best companies, they all start in the same place. They learn to measure what matters. And it
26:57turns out user interfaces absolutely make the hugest difference when it comes to intelligence. When you think about HCI, it affects like not only how the models perceive what you're doing, but also how the users perceive how you're doing. And so harnesses, the things that are running the loops, the things that are your billing agents on, uh, they they have to understand your context really well, right? And we we see the difference between if you switch out one
27:25harness to another, it affects the quality of the intelligence.
27:29The way that we tell customers who are working with fireworks labs how to think about this is you have to start with kind of what sort of data you have and to sort of understand is it sort of like hunch data like humans working on it kind of need to go like well I I couldn't tell you exactly why this works. It's a little non-deterministic, but like my gut tells me it's this.
27:52Well, that's a great place to be using models and it's great place to be using AI, but that's going to be a different kind of set of techniques and things versus the things where like you have a for sure deterministic results. So, that's your jobs of the world, right?
28:06Those are the things where it's like, hey, all my receipts, they always look this way. I need it to look this way. And so, that's a different kind of really solid evals and things like that.
28:17If you can measure it, you can improve it. That is like the great secret to AI and like learning to find what I call the seams of the work that you're doing is how you get there. Don't think of it as one problem. AI teaches us to break it down. It's lots of problems and then to understand what's measurable and what's not. Where do I need my hunches and where do I need my facts? And the of course the more facts you have, the better you can measure it, the better you can do. My favorite example is in
28:46the field of video compression because it used to be that video compression like there were so many people who got their like doctorate in electrical engineering on video compression and for a while it kept getting better and better and better and better but it finally petered off. Nobody thought there could be any more intelligence that was there. Nobody was like signing up to get the next PhD in video compression. But with reinforcement learning, they were able to find four to six% more video compression that was on
29:16the table than the best people could find simply because it was so innately measurable. And so I want you to think about for whatever field you're working on, if you can start understanding your data at that level, if you can break it apart into small parts that can be verified, if you have something that you can score, then you can do better. No matter what it is, I don't care if it's customer service. I don't care if it is, you know, a generative medicine, all of these things, if you can measure it and you can start building that data
29:46practice, you can get it better.
29:48One of the things I keep seeing and hearing and and certainly seeing on in my own practice, the way I use these tools, the way I'm seeing other people use these tools, it absolutely amplifies the best parts of your practice, but it also amplifies the worst parts of your practice. Um, and it reveals some of the the cobwebs.
30:10And I'm seeing a lot of things when you're talking about improving, finding measurable improvement. I find people absolutely realizing that, but finding it at such local points that it doesn't add up to anything. You're creating a tangled spaghetti of of a bunch of little improvements that don't actually work together. So, how are you what's the best practice that you're seeing out there? What's the secret for teams that are and companies that are executing well? Is it they had really solid best practice in the first place or they're
30:39adopting it as part of this? What where do you see people doing it this well?
30:45So, it used to be during the design process that you know we would have like wireframes and we would have these other things. Now I have my my best designers and product managers. They will come with like a prototype and that that prototype like you can actually click it. It looks like it might be doing the thing but probably like one path is actually working. But
31:15then what happens is the quality of the conversation goes up so much from that experience of having that prototype stuff because it's like oh that's what you meant and then you have the real conversation. It's like what are all the ways that this could fall apart? How do we start talking about what matters in this thing? So the it used to be the first step was a design that like it was a little upshot and people didn't quite understand but now you have something that people can really react to and have
31:42real conversation around. The worst companies stop at that stage, right? You know, and we and we see it.
31:50We see uh like done is the most seductive word I think in the English language where it's like no no the that's just can't we just do this?
32:04You know, you put out this little prototype and people like, "Oh, this kind of works." And you're like, "No, this was vibe coded."
32:15That that that was the first step. Now it's time, right? The 50 or 60 iterations to get this from a prototype into something that's going to be meaningful. And so I think about it as that gateway to these conversations. Now the tricky bit and it's always a tricky bit is you know it's very tempting to show a prototype to a customer right it is very tempting to go in and
32:44show something that is like looking pretty done and there's a lot of expectations that's coming to to the fold and I think it's so much more important that you're actually talking about what done is before you're showing that prototype if you haven't had a conversation on like how are you going to know if this worked? Then you're you you skip the most important step. And in AI, I think that's always the step that
33:12uh it's it's it's not that fun to be like, how do we know when you're done?
33:18Um, well, the the the conversation I keep having with people is that first 80% of product development, of getting to a prototype, of knowing what it is, being able to describe it, not just a written PRD, but something that people actually understand that's got real definitions behind it. That's so much faster. It's so much easier because we can do it, we can feel it, we can have a much higher quality discussion about. So discovery is definitely accelerated. The problem is that second 80% of making it reliable
33:47and shippable and and uh something that's going to deliver value consistently. That's still really really hard. How do we accelerate that? and and that's the seams and that like I know it's a weird idea because there's always been seams in our work but sometimes like there's these these this these kind of funny valley executives where they'll say things like oh if we're going to we're going to replace this whole function with this and then they don't
34:17realize that inside of that is you know a million different jobs but then like the person has kind of figured out it's like well if we were priced one of the 100 things that needed to happen there but not realizing what what was in the work. And so you have to understand the things that are within the work. But then also as I said like I I think that it was very easy
34:46to look at something that feels like it kind of got like oh it kind of worked and it's like it's so seductive like well maybe I could just put that in there. Um of course lots of things are going to kind of work but that's not the work. And so actually when I when I talk about this I think of the the history of software and how it got here in the old days of software.
35:11What was software like literally it was was not hardware. So but it was the things that you were selling. Software was the things that you were selling that were soft and not hard. Um and then we had software as a service. And software as a service really switched where it's like hey I'm promising to keep this up to date for you. I'm promising to keep uh this information always lively. So you come to here, you're getting this service on the other side. But now we're at work and work is
35:38honestly like you have to actually understand the other person on the other side of it. You are now literally selling the work on the other side of this. And it's it's kind of the most human thing ever cuz we're always building these systems for other people and like those relationships and understanding what's on the other side of it and the expectations of how to talk about quality and if this worked and setting those things up in the front are are so much more important in this
36:06era of work than ever before. Which is why we see forward deploy teams everywhere. Right? It used to be software was mostly like, okay, we're going to send this off into the think tank and we're gonna write down this whole list of requirements and everything else and then we're going to ship it to you iteratively over long period. Now, we're actually getting in there and really trying to understand along those work, what's that hidden contract on the other side? What does the data look like? How does this go
36:35through together? And I don't think it ever goes away. I think we'll forever be in the era of work where it's taken a long time to get to the point where that other piece of it, that craft to suffer, the barrier to entry wasn't so high that like it can finally be about all the design behind it, understanding the other person on the other side of it, understanding the data on that thing and not just uh kind of saying here's the stuff and you're going to have to figure
37:03it out because this is the way our system works. Now the system actually has to work for you.
37:09So for a while all the AI CEOs and and everyone all the prognosticators were talking about how the world is going to change and we're going to be able to do everything with companies of one and things like that. And obviously that is not where we're going at the moment. But I'm also hearing from I'm hearing more engineers than ever.
37:28Yeah. And one of the things I'm hearing from CEOs and CTOs is people individual people are being way more productive but they're also running to stand still. They're they're just trying to keep up to date and keep things solid and running. We're not seeing huge advances in new features and great new experiences and amazing new products.
37:47We're seeing, you know, point solutions and some new models and things like that, but we're not seeing revolutionary new software coming out or am I just missing this? What do you think is happening?
37:59So, I I do think there's a bunch of revolutionary new software, but of course, hyonic adaptation. the second something comes out, we've already decided to start out this amazing agency. Fair enough.
38:10Um, but you know, uh, my my mom passed away from cancer like a decade ago. And when she was, uh, when she was losing her ability to uh to type or to be able to see, like I had to work my hardest to try to figure out how I could set up an iPad where she could send letters to people. Um, and the the technology was pretty painful, but we figured out a way
38:37that it could like slowly do this. Now I can operate my computer with things like Whisper Flow. I can I use voice throughout my day all day to do work. I record myself while I'm on my drive on the way to work. I'll like set up AI agents to research things. So I'm like, "Okay, today I was talking to these people and we were talking about all these things I don't understand. I need you to go find first resources on all
39:06these things. Summarize it for me." And then on the way back from work, I'll listen to a generated podcast of all of the research that I was listening so I can kind of start learning this stuff.
39:18That's incredible, man. Like you, we you have all of this technology where you have access and the ability to go and and mix and match resources and work on many more modalities of experiences across all these things where you can be so much more expressive than anything that we've ever seen before. And you can get people to be able to uh communicate and receive it in the way that they like it to be received. You know, you
39:47mentioned that you used to be in uh the music industry. Uh I I my undergrad was in music and computer science and I studed music technology for AI. And what we hoped was that we're going to have so many more people who were going to build their own music because we were going to reduce the barrier for people to make music. because people have these incredible ideas
40:14with the like they can like hear these beautiful ideas of sound but might not have the technical skills to be able to do so. I think it's so democratizing that now that we can have more people creating richer more interesting experiences without having those incredible gaps to be able to do this. And that's the my future I think.
40:38The interesting parallel though to to me is as someone who was worked as a as an editor and in music the most important thing was not the ability to produce the music. It was the uh ability to edit the outcome and and curate and choose what to actually publish because the most incredible artists whether they're visual artists, audio artists, written artists, whatever, they produce way more than what you northeast
41:07and the you know when you get access to their entire archive it is not always a good thing. Um the best stuff is incredible, but there's a lot of other stuff that is nowhere near at that quality. And you know, just producing volumes and volumes and things isn't always the answer. Curating uh and finding the right stuff and concentrating on that is is absolutely a skill as well and is is going to be one of the most important skills going forward. You
41:36know, I think of all of the new all these new fields that open up in this area. Of course, now we have AI ethicists and we have people who are themselves AI artists. They're going through their remixing. They're finding all these new modalities and everything else. But then you still have to have critics. You have to have the people who go through and help other people understand like what is pushing the boundaries and what else is interesting that is out there. Yeah. Unless you're a
42:04Shopen or Bach, I think the the full archive probably doesn't need to be opened. Um, you have to have a few practice runs in there.
42:14Yeah, these are tools. They can be used really, really well. They can be used poorly, but you know, we already went from uh changing with Garage Band itself changed the the way that we record and produce and and distribute music itself.
42:28So, Rob, this has been fantastic. I've got time for, I think, just a couple more questions. I wanted to go back to Moes very quickly and talk about from the perspective of a product team and also a management team. So a product team comes to you and says, "We're terrified of being disrupted. You know, people can copy us so quickly. What's the one thing we should focus on this quarter to build a real moat? What what kind of response do you have to them?
42:56People are going to copy you, right? And so that your product doesn't stop at your features, right? So you have to be thinking about the entire experience and you have to also be thinking about how are we going to be distinguished with our data and and AI the the most important thing you'll notice actually who are the AI companies who are getting the the most funding right now it's actually all the ones who are data focused whether it be
43:24you know even a very small set of very curated data can be used to specialize intelligence specialized experiences and so the very best companies are going to be going through and they're going to really understand their data. They're going to really understand now like and at every single scene, you know, here's the way my customers expect data to be to be coming in on the other side and here's what makes that a good experience. Here's how they're going to verify it. but all the way at the the
43:53actual HCI of the experience. You know, if you take a screenshot of a good a well-designed piece of software and you put that in to is your first prompt, you will get better intelligence from then on out than if you do the same thing with a a bad screenshot. They'll give you that exercise to do at home.
44:15But because there's context that is hidden in how the UI is arranged whether it be the validation that has happened there the AI can infer that right whether it be understanding kind of what we think is important in that situation.
44:32I think a lot about uh the words that do work for your organization. And so I my my kind of canned example is I you know I play piano and a a string on piano is a piece of wire.
44:45So every time I'm talking about string I'm talking about a piece of wire but you know my intelligence needs to know I'm talking about wire and not yarn and makes sense right? You have to be able to work with intelligence where you actually feel like it's getting smarter with you rather than stupider.
45:03Um the the state-of-the-art for most people is that they switch models and the next model that they use like as they're using it as there's more and more context it's getting worse and worse and worse at its job. could it's it's a reverse persons right a person the more context I have the better I get at doing the thing the more curated the more exact I become the these intelligences that people are not going through and building that context they
45:31get worse over time so so the basic skills of product development being able to figure out uh an audience who you can deliver real value to demonstraable value with the with something that you have proprietary access to or and and make it a good experience. That's that's really the key.
45:52Good data, good experience, understand who your customers are, ask great questions. And I think it's, you know, it's all these same fundamental product skills that you had to learn on the way.
46:04It's like kind of the golden era of the products manager in a sense, but it's only the golden era of the product manager if they can avoid that really seductive word which is done because all the pressure in the organization is entirely going to be around that word and if you if you can't push that conversation to the forefront. Okay, but how how do we know when we're done?
46:28because otherwise we're gonna have something that works for like well with one of the it's one of the things but you're not going to have done in a way that your customer wants it to be done and I think I will I'll caution folks that people really underestimate the value of their data. uh AI labs these days, especially the closed ones
46:58are really want to discount what your data is worth for them to absorb it. And I will say that we tried to do the math on how much of a discount would we have to give to our customers if we were to like start train their data because fireworks is zero data retention. we don't keep anything unless you're specifically using it to train a model and that but then it's it's yours the whole time and uh we we we couldn't figure out the
47:27math because it was just it was like it's a lot. That's the answer. It's uh your data is incredibly valuable. How you work is incredibly valuable. And I think that people greatly underestimate uh what they're giving away for free to make everyone else's intelligence a little better.
47:47Sorry, just going back to what you said a moment ago, it's yeah, build, measure, and learn hasn't changed at all. How we do each of those steps ch has changed radically, but it still comes back to that that core concept. Um, but let me ask you the last question, Rob, at a management team layer, you know, at a senior management layer, what should they be thinking about with relation to Moes and what kind of timelines should they be thinking about? Is this you know we used to talk about three to five year plans and 10 year visions and then
48:16thinking in quarter and annual plans what's changed at the management layer.
48:22Gosh, I wish I had a a fiveyear plan.
48:26That would be phenomenal. Um, I think my boss probably does, but but uh, you know, it I would say it sort of there there are certain things that you're going to be able to count on in terms of the way that intelligence is working and those things that you can count on kind of help. And a lot of businesses themselves have a seasonality to them and so a a lot of the folks that I'm advising the the the core basis of the seasonality of
48:55the business is just kind of what it is. If you're in education, they buy once a year, right? Like it it's just going to be what it's going to be.
49:03Um I think you have to restructure your teams themselves and th those those teams need to be composed of sort of different things where the trend here is really clear. You you have to have a much more direct relationship with your customer on the other side. You need to be talking about what the work is. And so I think the the the big difference between kind of old worlds and new worlds is that like our our teams are much more forward deployed. And I think the end
49:32result is if you if you don't have teams that are actually directly engaging with customers and trying to build a superior product for those customers, then I I I think that's going to be the biggest fault that you're going to see on the other side. Um, a lot of folks want to sit in the design pantheon and have it be entirely from themselves. And I think that's going to be a little challenging uh because it's
50:02uh it is far far far too easy to copy and the more durable thing is a relationship that's on the other side.
50:11Rob, this has been a fantastic conversation. Thank you so much for taking the time out with us. Yeah, thank you so much for having me.
50:19The product experience hosts are me, Lily Smith, host by night and chief product officer by day, and me, Randy Silver, also host by night. And I spend my days working with product and leadership teams, helping their teams to do amazing work.
50:35Luron Pratt is our producer, and Luke Smith is our editor.
