0:08First off, it's been absolutely amazing to see the speakers before me. I want to give one more round of applause for the incredible amount of content and preparation.
0:17I'm inspired. I've written my speech three times. I learned how to do storytelling. And so, here's a story for you. At RAM, we're known about velocity.
0:25We love speed. I love speed. I love speed so much that I once signed up to race cars at Le. I'm just kidding.
0:36Lemons. 24 hours of lemons. It's a race where you buy a $500 car and you race around for 24 hours hoping to win.
0:47Unfortunately for me, my car did not make it.
0:56I crashed hard. I was okay. The car unfortunately was not. Now, this crash was entirely my fault as the driver. But in real professional racing, the driver is rarely the problem.
1:11In fact, the driver is only 15% of the impact on the race. The real impact is the interaction between the driver and the car, between the car and the team.
1:23In other words, winning isn't just about driving faster. The best drivers in the world can only perform up to the level of the system. Winning the race is about removing the bottlenecks around the driving. That's the key point I want to drive today. Take for example the pit stop. In the 1950s, it took 67 seconds to change tires. Today, 1.8 seconds.
1:52How does 67 seconds turn into 1.8?
1:56Certainly not by asking the mechanic to work 37 times harder. All you leaders, you know what I'm talking about. They found the bottleneck and they iterated to remove them. Specialized functions, better technology, more practice. And F1 is relentless. 90% of the parts of a car change every year.
2:18only 10% of the 16,000 parts of an F1 vehicle carry over to the next year.
2:23Now, I want you to also think about what if 90% of your code changed every year because that is the bar. Why am I telling you all this? We're in a race.
2:35Speed is everything. Execution is everything. And we're in a race to build with AI. For the race car, it starts when the car comes in for new tires and the clock stops when it leaves. For us, the clock starts when there's pain and it leaves when the customer has a product that solves it. AI simply removes the bottleneck but moves it. And the best team, the winning team is the team that can find the bottleneck faster, remove it, and move on to the
3:04next one. Now, we all know the product development life cycle. We identify, we define, we spend a lot of time building and then we improve. Now the engineers, fantastic engineers, they've been able to automate a lot of their work and coding is now much easier. So here's a tip. Be more like engineers. Be as lazy as engineers because the reality is that the bottleneck has now shifted to us. We
3:33need more things to define. We need more things to collaborate. We need more things to coordinate. more things to ship, more things to test, more things to release. The bottleneck has shifted to us and so we need to invest in our own factory. The speed of this loop determines the outcome of the race and the question becomes, how quickly can you identify the bottleneck and redesign the factory around it. What I'm going to show you today are five of these steps and how ramp has gone far in iterating and automating through it. The first
4:02step is identify identify the problem. Identify the pain, right? The issue is that that pain lives somewhere and there's a lot of pain, a lot of data. Gong, zenesk, log, rocket, a survey, very angry email to an executive. The problem is that context lives all over the place. Too many silos, too many opinions. And so the first bottleneck to identify is how to sift through the noise. And LMS are great, but a 1 million token window,
4:31that's less than 0.5% of gong transcripts at ramp. So, we started small.
4:37We created a hate channel and it every day it posts all the lovely things that our customers say. And yes, these are real quotes. [snorts] That got out of hand really quickly.
4:51[gasps] Yes, I know. I am not proud of it. No one's perfect.
4:56And so, we had to redesign from scratch.
4:59We built an actual customers insight agent. That agent pulls from all the data sources at a company. It uses traditional ETL and pipelines. It uses actual vector search. It has clustering around the same context. It understands our product. It understands our teams.
5:18It understands the features that we have and it's accessible to the rest of the organization. Once we had this working, we asked ourselves, how do we get this to as many hands as possible? And so we again experimented a Slack agent where you can ask questions, a nice HTML dashboard that people can log in and read. My favorite one was the podcast, the hate podcast.
5:43Great way to start your day. You listen in 100 customers yelling at you about all the ways your product is broken. The concept is simple, right? Get people access to this data as fast as possible.
5:54That doesn't mean don't talk to customers. It means you actually identify exactly which customers to talk to because the data is traceable. Okay.
6:01Once you've found what to focus on, the next step is to define your product. How do you go from a concept to a fully scoped idea? And so we turn to AI, right?
6:15And the AIs all kind of look the same.
6:17They ask you, "What do you want to build?"
6:20And you stare. You have this super intelligent being asking you what you want to build. Is that really the best question to ask? Is that really the best starting point? Right. The goal with AI is to connect it to your systems. That is how you're going to make AI extremely extremely actionable. And so we did that. We built our own AI agent called Glass. And we gave it all the context it needs to do the work.
6:44It connects to all our systems. So it understands both the data with Snowflake. It understands our user research and it can nail exactly the job to be done that our customers are asking for. That gives us specificity with both qualitative and quantitative data. Glass also understands our product strategy.
7:02It understands how we define product specs. It understands our codebase. And so instead of a PM bothering an engineer asking them, is this possible? Would this break something? What am I missing?
7:13You ask AI. AI can be your best thought partner. So now AI is your tech lead.
7:20And finally, you can build a prototype because it understands our product principles. It understands our design system. It understands our codebase. You can actually define and build a prototype that actually works within your product. Your engineering team don't want any of these individual things. They don't want your prototype.
7:37It's useless. They don't want your long spec. That's useless. They want the combination of qualitative and quantitative data to convince them that this is a problem. The actual requirements that they can use for their coding agents and a prototype that they can actually get inspired by. That is the next contract. So now you have a strong foundation for which to go. The next step is to build. I mentioned that building is no longer the bottleneck.
8:03Obviously it's a bit more nuanced than that. When we focus on one bottleneck, the next one appears. The first bottleneck was coding. Everyone should have a coding agent at this point. We built Inspect. Inspect is our own coding agent. And the re reason why we built it ourselves was because we wanted to have a strong harness and we wanted to make sure that our coding agent really understood our codebase. It works inside of Slack. It's fully provisioned. It can run under 5 seconds and it can return an
8:30actual product in a deploy preview so that the actual product manager can actually use and interact with the code that's being built. Anyone at RAMP can now code. Inspect has a million sessions. 75% of our PRs is actually built by Inspect and a thousand of those PRs over the last month was submitted by a non-engineer.
8:52One tip here though is that coding agents are really good when you have a strong architecture and a strong code base.
8:59This expands how much of your company can actually use AI. You now are shipping a lot of code. The next step is code reviews becomes the next bottleneck. Engineers are just constantly combing through a ton of code. And so we focused on that. We built review buddy. Review buddy fully understands our codebase, fully understands our actual uh quality checks, our security concerns. It finds the right human reviewer to loop in and it also understands the context, all the prompts that led you to build that code.
9:27So it can actually audit how you used AI in the first place. So the review and the collaboration is now grounded in visible context. 93% of our PRs are now automatically handled by review buddy. So that the highest quality engineers are spending time on the 7% of PRs that truly matter that have a lot of questions around are we building this correctly? Are there any risks? The next bottleneck is testing. Previously PMs,
9:55I'm sure you're familiar with this. We spin up a QA environment. We try to tweak different fixtures. We click through the screens. We make sure that the engineer didn't skip out on that one feature we really liked. We automated that. We built Testo. Testo is a browserbased QA agent. It spins up the product in a 100 different combinations based on the actual production data that we have access to. and it runs in the product like a user. We give it instructions. We say, "Hey, pay an invoice,
10:24but amvertise it." And it clicks around and actually gives us feedback. Both blocking feedback, bugs, issues, blockers, and honestly extremely thoughtful design and qualitative feedback around things that were confusing, things that break our design system or breaks our design language.
10:40And that goes back into the product development life cycle. Testo in the last 30 days caught 425 bugs.
10:48I think these are bugs better caught by us than our customers and they send it back through the loop to fix. So now you're shipping a lot of product.
10:57The next step is to coordinate. How do you make sure that everyone in your company is in the loop? Things are moving so fast. You're shipping so many different things. The next bottleneck becomes human attention. We've all been there. PMS being inundated with notifications.
11:14And so our job becomes coordination.
11:16I see a few people coordinating right now.
11:20Too much process, right? And slows down the builders. Too little process and everyone is confused.
11:27And so human attention becomes that bottleneck. How do you solve this? Well, every question is an API. Every question is an API. And you architect a system.
11:39Gadget essentially understands the intent of these questions and connects it to the formal record. The road map in notion, the specs in notion, the customer calls in notion, slack, linear, our tickets. The principle here is simple. For you to empower an agent, the agent needs to be able to understand and read the organization. And the organization needs to be legible to your agents. I'll give you an example. A question is like what's the status of
12:08this project? What's the status of this launch? Are we on track? An agent handles that. It understands the full picture. It understands who owns the next steps. It answers with evidence.
12:17But not only that, the agent updates our road map. The the agent gives status updates. The agent pings people who are late on their deliverables. There's a lot of fun ways you can actually scale.
12:28Or another question, sales, asking questions about, you know, what is this feature? How do I sell it? Is it available in Brazil? What's the use case? What's the price? All of these things are fully understood. And Gadget, if it does not understand, routes it to the right person. But we can take a step further. A launch. Gadget understands the product. And so it can write the help center article. It can help write the blog post. It can help write the customer email. So on and so forth.
12:57Again, start one place. You'll see how far it expands. 85% of questions that are being asked to PMs now are fully answered with AI and those that are not are answered and fed back into the system. So you're shipping a lot of products now. Everyone's fully aware and coordinated. Everything is perfect.
13:16But that's just the start, right? You are constantly improving on the product and the loop starts again. Now PMs, we we naturally gravitate towards the small reactive things, the things that we have high confidence in, the things that are easy, that are visible, the dopamine hit. Oh, we've solved something. I think that is a huge mistake. Claire, I think, had a great talk around how to be ambitious. You don't be you're not ambitious because you're focusing on small things. You need to automate your way out of these small loops. And so we
13:44did just that. For most small things, an AI fully runs the loop. It routes it to the right team. It matches it with your backlog in linear. It ddups. It counts.
13:55It ranks. It runs the plan. It has a human in loop through Slack around, hey, is this ready to go? I'm ready to code.
14:02It writes the code. It runs through test and CI/CD just like we talked about. And when it launches, it runs through the knowledge base. These are small loops.
14:11And humans are in the loop only just barely. Thousands of little loops with thousands of little things so you can focus on the big ones. Thanks to these autonomous loops, 60% 60% of UX issues that are identified by either a customer or a salesperson or a CX person or ourselves are fixed within 24 hours. So, at this stage, you're probably asking yourselves three questions.
14:35The first question is, okay, how do we know we're moving faster, right? How do we move faster with quality? After all, you know, we talk about product velocity. How do you measure it?
14:45certainly harder than, you know, a lap time at a racetrack. Well, some things are not easily quantifiable.
14:53Nikki Laa in 1974 took a lap with the Ferrari car and told Enzo, "This car is a piece of It drives poorly. It handles poorly. It breaks poorly." Got out of the car, went to the engineer, fixed it. That is not dissimilar to the culture at RAMP. I'll start there. But the point I'm trying to make here is that you you will know if you're moving fast enough if you actually hire a driver that knows what speed looks like and you put them in charge and you
15:23actually allow them to challenge you.
15:25That might be hard for some leaders, but that's what we need to do to be challenged by people who know what great speed looks like. The second question might be what about our budget? We have limited resources. We have limit resources in terms of engineers or simply with tokens. How do we compete with companies with no constraints? In 2006, Audi asked Audi had a bad car, a very slow car. And so they asked themselves, how can we win if we can't
15:54go faster than anyone else?
15:58Their answer, fuel efficiency, fewer stops, more time on the track. And they won LA three years in a row. So constraints force you to choose a dimension in which you can be world class. So embrace your constraints but not your bottlenecks.
16:17Find the one thing that you're really really good at. Find that one bottleneck that you think can 10x your company and just start there. And the last question, something that we've talked a lot about today is the future role of product. Are we automating our jobs? What's going to happen if these loops are running without PMs? Well, the first thing I'll say is I think it's amazing for for product managers to be out of the loop for a lot of things that engineers can drive. This helps us get a ton of
16:46leverage and I don't think we should be scared about that. Our jobs though I think will evolve in three ways. There's three tracks that I see for product managers. The first track is the technical track and something that I don't think we are investing nearly enough. It's the PM that's building the actual factory, the technical product manager that is identifying bottlenecks and removing the drag in your organization. They're not shipping products for the customer. They're shipping the product that helps you build the product for the customer or
17:15helps AI do so. The second role, arguably one of the most important roles, certainly the one that gets the most limelight, is the taste maker. It's the driver. Ultimately, you need someone who's holding the steering wheel and doing things that AI cannot do and holding the bar for what great product, great taste looks like. And finally, the GM.
17:36I think PMs will absolutely evolve and expand their role to not just the product organization, but the marketing organization, the sales organization, the growth organizations, the operational organizations. PMs will become GMs and they will own the actual business outcome and lead the entire function. So, we walked through a lot I'll recap three main takeaways. The first is that speed is not about just coding faster. It's identifying and removing the bottleneck in your build
18:05process. And everyone has one scientifically. There is always one two as you as you actually saw that first bottleneck, the next one appears.
18:16And your job is to move through them as fast as possible. And lastly, us as product leaders, we need to obsess a little bit less about the product that we're delivering and a little bit more about the factory that helps us build products faster. Now, I just walked through a lot here, a lot of great things that we built. And I did so because the reality is that everything you've seen here is outdated.
18:41Just like F1, teams will copy each other. I want you to copy us. I want you to actually hopefully surpass many of the things you see today. And I'm sure people here have gone much further. The reality is that it's all about the next season. And so I'll leave with this.
19:00Enzo Ferrari said, "The best Ferrari ever built is the next one. The best product you'll ever build is the next product that you'll launch. That begins in the factory. That begins in the software factory that you built today. I hope you copy us. I hope we get to copy you. Let's share more together as a community around how we are building these products so we can deliver more for our customers. And I hope you all stay stay safe out there. Thank you.
