0:05Welcome everyone. Welcome Guillermo.
0:08You know Guillermo, he also goes by G, but I'm going to practice my Guillermos this whole time if you think that's okay with you.
0:16All right, move fast without breaking trust.
0:20Not easy to do, but let's start um with uh with a story. Your one of your favorite movies is um is Pixar's film Ratatouille.
0:33by like why do you like that movie?
0:37Oh my god, there's so many dimensions.
0:39First of all, I just love especially in the age of AI, I heard in the previous fireside we talked about quality.
0:48I think something that distinguishes some of the finest creations, some of the finest pieces of art, movies, everything is just quality. I think Pixar really embodies that. It's something we strongly believe at Vercel. We're just chatting a lot about how with Paul from Salesforce Ventures, we're talking about the killer app of AI in my in my view is coding.
1:13Um and one of the dark sides of AI is that it's so easy to go zero to one that you might just be doing it too much.
1:24And that picking the right battles and choosing what to build, what to generate, and maintaining a high level of quality is going to become a differentiation. And I ultimately think quality begets trust. What is trust?
1:37Especially with a software business and software company like Salesforce or Vercel, just quality of customer obsession and service over proven over time, that's what I think trust is. To me, Ratatouille embodies that. Pixar embodies that. And also, it has this hilarious part of the movie or theme, it's just this saying that anybody can cook and encouraging people to try things and build things.
2:03And and I think AI is helping us all do that, right? Uh obviously talking my book because Vercel helps everybody within the organization build applications and deploy them uh securely. Um but uh I truly believe that AI's is giving us all superpowers and that is truly a killer Yeah, let's talk a little bit about quality. You know, I I just uh hired someone to take my job running Slack engineering, but when I was running Slack engineering
2:31you gave us a very hard time. You had a lot of very specific feedback on very detailed things about because you use Slack. You use Slack heavily. You're a big part of the Slack on Slack.
2:43Um but you had a lot of detailed feedback and I would equate it related to quality. Your expectation, your bar for quality. I think Slack has a lot of quality, so uh it's a very high-quality product, but your bar was very high. Um so talk to me a little bit about that where and your expectation of speed also.
3:02Like you were like, "Why isn't it fixed?
3:03Why haven't we done this? We have to go fast." So maybe give me a little aspect of quality and speed.
3:09You know, as a spokesperson for a lot of engineers in the world, the question that I get very frequently is you know, as an engineer, am I going to have a job in 5 years, in 10 years, in 15 years?
3:20Creating high-quality software is extremely tedious and expensive.
3:25Uh I actually give it Slack is a great example of something that on the surface seems like you should be it should be easy to generate something like Slack cuz you look at it it's like, "Oh, it's a sidebar, channels, conversations. How hard can that be?"
3:38But the depth of quality of engineering around security around scale.
3:46I remember when you guys Slack uh went the vertical during COVID, right?
3:51Supporting the throughput of messages per second like so there's all of this dimensions of quality, scale, security, reliability.
4:01Um and what what we try to do at Vercel is your next incremental piece of software has to always be higher quality than the previous one. You know, we we've been excited talking a lot about AI software factories because as code becomes free the value or or the the differentiation of a firm shifts when they are up into the best practices, the institutional knowledge, the context engineering Yeah.
4:32that you put into your factory to produce that next incremental piece of software. For example, something that I care deeply about that you just called out is speed. And over the years they collected a lot of nuggets of wisdom about speed. Speed is not just, you know, I measure a transaction in number of milliseconds. The speed can also be the perception of speed.
4:54Uh a great example, I give you guys a lot of feedback cuz I travel so much. I'm on a plane all the time. Occasionally a plane will be Starlink enabled and my happiness level goes through the roof.
5:07[laughter] More often than not uh it's Gogo in-flight or whatever. And what happens with these internet connections is that they drop a lot of packets. If they drop a lot of packets, it makes messaging your friends, family, and co-workers really, really hard.
5:22And so what do you do in those circumstances? You have to provide a pleasant experience to the user even though the network is unreliable and slow, there's things you can do to make that experience more pleasant. Now, you know, I've been studying real-time systems uh and uh human-computer interaction for decades.
5:41Now, how do I transfer that to the next generation? How do I enable my co-workers to use AI plus these best practices. So, we work a lot on codifying this into skills, into frameworks, and into tools that that make this factory work better.
6:00Yeah. Well, it's kind of a in our keynote, we talked about determinism and probabilism. Hard to pronounce.
6:08Um Salesforce is definitely you know, really working on providing frameworks, uh you know, around the right context at the data layer, you know, the right application definitions of your business processes or semantic layer, the right definitions of AI. You know, because anyone can write code anything. These new interfaces, AI first layer as we call it, it can look amazing. Like, "Wow, that's that's incredible." But, it might be the
6:37wrong data, it might not be secure, it may not even be a very good interface. So, how do you think about um allowing like the power of building is now in everyone's hands. You're helping make that happen at Vercel, which is incredible. But, how do you think about giving that power but adding those guardrails to guide, to security, to quality of user experience, to all the things that you I would call trust.
7:05Something that has created or enhanced the trust that our customers have for Vercel is our care for design.
7:15We care a lot about the craft of our interfaces. We care a lot about the consistency of our interfaces. And so, if everybody can cook, can they actually all design beautiful interfaces? Well, we run the experiment and the answer is no. You ask a an agent, "Build me a website or build me a sales deck." And they all look the same. It it's it started out with purple gradients.
7:40And it doesn't look like Vercel.
7:42look like Vercel. Started out with purple gradients, then it became beige stuff with serif fonts. And you look at it and say, "Well, it's really cool that we can do this." By the way, I don't dismiss that at all. Like, it's amazing that anyone can transform prompt into artifact.
7:58But, how can you infuse the brand and design principles of your company? So, the first thing we did is we we took this standard that Google pioneered called design.md. It's basically a skill for design that brings some more determinism. So, I really like that framework that you just put out, combining the good parts of AI, which is the creative stochastic nature of it, with some determinism. So, design.md, for example, can codify your color system.
8:28You know, obviously we like black at Vercel, but we like some other colors and like we listed them out. Um and then you can actually use some more stochastic stuff. Like, you can write down in English, "Hey, if I need to produce a report for sales, avoid this and avoid that and use my tone of voice here and there." So, design.md helped us a lot, but we took the deterministic guardrails even further.
8:53Um yesterday we announced a linter for enforcing design system constraints.
9:01So, one of the um and this is just an example of like the the cool things that you can do, but I think the future is going to be combining these two things. I call it verifiers and skills. Skills alone are just guidelines. And just like humans can sometimes mess up and not follow guidelines, you need a hard verification system. You need something to enforce constraints.
9:24Uh in engineering, you know, the the there's a programming language Rust that is growing in popularity because Rust imposes very harsh constraints around memory safety. And and so combining the loose of AI, the loose vibes of AI with hard almost like mathematical proofs is I think the future of of a lot of this um where software engineering is headed.
9:51Um have you heard about the SAS apocalypse?
9:55[laughter] Um well, we've been talking about that that it's not have played a role.
9:59It's not really true, but I've heard you talk about like you're not going to vibe code all of your systems in the enterprise.
10:05100%. So first of all, I think that discipline to decide where you put your tokens. So I think the first wave of AI was CEOs. And for the record, I didn't do this, but CEOs were holding like leaderboards of like, "Please just spend as as many tokens as you can."
10:23Token maxing. Um I always thought it was the dumbest thing ever because fundamentally, tokens are compute.
10:31Would you ever wake up one day and say, "Hey everyone, uh go to the AWS console and create as many instances as you can." Like fire up as many computers as you can. Well, I would fire that engineer saying like my costs are going to go to the to the floor, right? And so with tokens, it's the same. It's just GPU compute.
10:47But we were for a moment comparing token usage to productivity.
10:51Yes, I think I think that's wrong. And I also think again like the question that we need to answer is there's some finite amount of tokens that you're going to use, right? Where do you put those Where do you put those tokens towards?
11:05Is it towards hardening your cybersecurity? Great use case. We launched DeepSec, a cybersecurity harness. I believe that that's one of the best things you can do with AI today is the strengthening the reliability and security of your systems.
11:21Agents with computer use and browser use can literally navigate your interfaces and your programs and tell you where you're messing up. So all of the things that sort of increase quality I think are great uses of AI.
11:32Now, going to SAS apocalypse, I think what AI is showing is that systems of record are extremely valuable, but they have to have the right agent ergonomics. So, agent ergonomics is the or agent experience is the term that I use for if a if I have a bunch of data in my CRM, can an agent access that data? Do I have the MCP for it to access that data? Do I have the CLI, the SDK?
12:00Uh is the data fast? Is it scalable?
12:04One of the things that we're going to have to confront is that agents will use compute far more than human beings.
12:11I'm a heavy user of Slack, but it let's say I send 1,000 messages per day, right?
12:17Uh or maybe that's too much, but I send a handful of messages per day. Just imagine how much more agents can do because they they never get tired. And so, they're going to query your systems of record like you've never seen before. I think you guys were sharing some data on this.
12:30You you launch an MCP, and all of a sudden the demand for it seems like nearly infinite. Uh and so, I think the SAS apocalypse is what it is about is customers are going to generate a lot more software. They're going to access a lot more data.
12:46And those repositories of data are going to become extremely important, and they have to have the right interfaces for agents to to be able to use them.
12:53So, let's talk about interfaces. You know, we call it headless. That's our strategy.
12:58And Claude for us with your AI for us is the our actually brand. When we launched Claude for us, you saw that. And the amazing artifacts, those are incredible. Uh Slack calls them surfaces. So, in the keynote, you saw that.
13:12How do you think about Vercel and Salesforce? 1 + 1 = 4. And what what are use cases for infinity, right? Because uh when I started Vercel, I realized very quickly that the value that we're were to the world was in the customer experience side of things.
13:29So, a lot of our bigger customers were saying uh great example, customers like Under Armour and Nintendo, they wanted to innovate on the customer experience side, but they wanted to keep using Salesforce as their source of truth and and customer uh data repository. And so, they combined Salesforce and Vercel, and they launched amazing web applications and and websites way faster than they could have done if they had to rebuild this thing independently from scratch.
13:56And so, that's why the 1 + 1 = infinity. Now, the limiting factor back then was if you wanted to move fast and create the next underarmour.com, you needed a DevOps team. Vercel solved that. You needed to create frameworks like user interface frameworks, etc. Vercel solved that with Next.js. The other limiting factor was writing code.
14:21Now, that's not a limiting factor anymore. We can use coding agents to generate these amazing interfaces, to launch new products. And so, I like I mentioned, I think this is a match made in heaven. Also, Slack code, I I I I mentioned I run the company on Slack. Slack code for me is is sort of the it was the missing piece.
14:43Because so much of improving a product emerges from conversation. Conversation plus customer data plus feedback that we get, and then we digest it together. The best software has always been a byproduct of human collaboration, so now it's human plus agent collaboration in one place. So, um really really bullish on where things are headed there.
15:04Yeah, glad you like that. Uh so, for those of you who don't know, Slack code is it sounds like coding in Slack, and it is, but Slack's not offering the coding, so partners like Vercel can actually use the interface of Slack for multiplayer coding, and get all the value of Vercel plus the value of of Slack.
15:21value of again like the human oversight is still really important, right? The one of the most popular words now in AI is steering. Because what happens is the agents are working for longer. It used to be that I mean it seems like forever ago, but just a few years ago all AI for coding was was auto completion.
15:43Copilot inside your code editor, you start typing, it auto completes, and our minds were blown. Later we got ChatGPT, and we could ask it and you could build something and could you would copy paste the code. Now agents can work on code for hours.
15:59And that happened this in in just a few years, but they still need oversight. They still need steering. And I actually find that the Slack interface is great for this because you and I can you know we might be debugging a customer escalation.
16:15We might be debugging an incident like resolved.ai just just probably spoke about.
16:20We we we could be brainstorming a new capability. And so I think that steering of the agents by the humans is still very important. I don't want to live in a world where it's just the agents and we're not participating, right?
16:32So so how do you think we need to evolve as humans in our skill sets to be better at working with AI at delivering that quality, the trust, without breaking trust?
16:44You know, there's a term that I've been using called agentic inquiry.
16:48So a big debate in my engineering circles is should we read the code or should we not read the code? Or even if we're reading the code for how much longer?
16:58And my answer was like I don't longer will you be able to Yeah, exactly. Like are we going to read the code in 3 years? Or are we going to treat it going to treat it as assembly?
17:07Are we going to treat it as zeros and ones? And my answer to that is we have to understand the systems. That may not mean necessarily reading the code, but we can ask the agent questions of the code. We can have the agent work for a few hours or minutes or whatever and say, "Hey, show me what the new architecture looks like."
17:27Or run a simulation.
17:30Or and a big part of what Vercel is proposing to the market is don't get locked into one model. Have the models debate one another.
17:39Imagine being in a coding channel and Vercel is there and Cloud Code is there and Codex are there.
17:45And let's have this conversation, right?
17:47Uh and Fable critiques Soul and Soul, uh you know, delegates to Deep Seek to save on costs. And so, I see a world of multimodality.
17:57Um and and the humans need to understand we do need to change, I think, some of our expect- We need to reset and update our expectations very frequently.
18:07Uh I think if you get too stuck in your old ways and saying like, "Well, I'm sure agents can't do that."
18:12That's a big mistake. Agents can do pretty much everything if with the right guidance, steering, and skills. Uh the other thing that I've learned that a lot of people are doing at Vercel is asking the agent what questions we should be asking them. So, the idea of prompting presupposes that we know the questions.
18:35[laughter] And so, sometimes you can lean on the agents to say, "Hey, agents, if you were CEO of Vercel, what should I be paying attention to?"
18:45We have an internal agent at Vercel called V that is connected to everything inside of our systems, Salesforce, Slack, uh our customer insights, customer feedback, X. And we can ask this system, "Hey, what are the trends this week? What should I be paying attention to? What are interesting things that people are chatting up about in Slack?" Uh and so, I think Mhm.
19:10trusting that the agent sometimes know better than us can be can be a challenging thing to do.
19:17All right, this is a bit of a different question, but um back to the human element.
19:22Do you think people should still study computer science in college and you know, or what disciplines do you think are going to be most valuable in the future? If you were to tell, you know, young people going to college right now what they should focus on to prepare for the world that we see now and and as well as what's coming. What what would you say?
19:44You know, I was um I was doing this chat with Paul Drews from Salesforce Ventures earlier and he was asking me about my early days of I grew up in Argentina. I taught myself how to code. And I remember the things that I would find exhilarating. Like compiling a program and getting it to work.
20:03At you know, I was like 10 years old. I was like, "Oh my god, this is the best thing ever." And I think of my kids. My oldest one is 10. Compiling a program is like, "No problem." It's just even deploying an application on Vercel that could scale to a billion users is literally one prompt.
20:20Like when you when you deploy to Vercel, the infrastructure is so well tuned and so scalable that my kid could prompt and ship an application that could reach billions of people. That was unheard of when I was 10.
20:34That that means that what I want for him is to now raise his ambition by a hundred times.
20:40Okay, you have these amazing technologies. What can you do with them?
20:43You got to get those billion users.
20:45It You need to get a 10 billion agent customers or I don't know what I what I'm going to set as his as his bar, but Yeah.
20:50I think he can go further in creativity. He can produce video assets. He can he he can design. He can do so many more things. Um and so I think the future of computer science will be harnessing all this power.
21:06You know, computer science in the past was punching cards, then compilers, then getting programs to run, then publishing websites on the internet. So, I think we keep making those things seem obvious and easy. But, then there's another, you know, big next bar to to to to accomplish. Yeah.
21:24All right. Um a lot of talk lately uh about cyber and, you know, the models and, you know, um or even bioterrorism.
21:34Um What should we provide as vendors? I'm not going to ask you your opinion of where the world's going. Um but, I think we all I I really liked um Jensen's who was Jensen today is like, you know, don't ship it if it's, you know, and come up with the right testing. But, what more can we do as as software vendors to provide the tools for the future for the risks of the future?
22:03One is we we published an essay called everything hackable will be hacked. It's very important people That's positive.
22:13Yeah, yeah, [laughter] I guess. But, there is a positive angle to it. So, we we really needed to get this message out into the world. The agents of today, you know, Dario's saying about, you know, the dangers that come in the future, I'm talking about the dangers of today Yeah.
22:30are far more real and far more serious than we we we comprehend, I think, in a lot of cases. And so, the ability for coding agents today to hack into systems far exceeds what anybody is talking about. And so, what we did is we uh we created a benchmark for cybersecurity. And we started testing models. And we started realizing that there's way more vulnerabilities than we can even like like have patches for.
23:05Uh and so what the the the message that we deliver to the world is start putting your tokens towards defending your systems. Start running the scans. Start patching the bugs because they're already there. Uh we're going to be putting more money and more resources towards hardening the open source infrastructure of the world.
23:26So, Linux. I'm a huge Linux fan. Linux changed my life when I was a kid and I couldn't get Windows to be a good developer platform and I switched to Linux when I was 11.
23:36And uh it it the I love Linux, but we uh we have found a lot of vulnerabilities. It was code written in C. It's not a memory safe language. And it was written by humans. So, it has a lot of humans, you know, make mistakes. And so, we're going to have to upgrade a lot of our global infrastructure. Um and and again, it's not about the science fiction skyline things of the future.
24:01It's like it's a problem of today. Um and so, that that is a uh a thing that we have to do ASAP. Um the other thing is what's fascinating about agents is that they get better the more power and the more data you give them.
24:19So, when ChatGPT came out, it was really awesome, but people were saying, "Can you write me a haiku about Salesforce?" And we're like, "Oh my god, it wrote a haiku." And then we realized, "Well, what if it has more context?"
24:32Oh my god, it gets more intelligent. And then we realized, "What if it gets more tools?" Oh my god, it gets more intelligent. And then we realized, "What if it get What if it gets a computer with an internet connection?"
24:43Oh my god, it's even better. And so, Agent IQ grows commensurate to the power and resources that we give it.
24:52And therein lies the problem.
24:54If we don't sandbox these things and if we don't govern how they access data, then they will create risks, and they will exfiltrate our data, and they will hack hugging face, and combine that with the fact that they're being trained for running for longer and supervised, which again can be a great thing. You tell the agent, "Hey, solve the Navier-Stokes equations.
25:19I'm going to go to the gym. When you When I come back, please solve that Millennium math problem." It can do that. But it can also do things that you don't find positive or that you expect to happen. And so sandboxing We're creating We created the Vercel sandbox, which is a way that you can build agents and sort of control what they can do. Yeah.
25:41We can You can put a firewall around them. And so this requires a lot of innovation, a lot of new infrastructure that doesn't exist yet, but we're working on it.
25:50The other thing you said Linux was created by humans, so there could have been a lot of mistakes. And then you didn't say this, but it implies that AI does not make mistakes.
26:00Oh, yeah. Makes lots of mistakes.
26:02clearly an AI was trained on all the AI makes a lot of mistakes, but I do want to say something positive about the AIs. Um because you can combine them with these verifiers. Uh colleague of mine called it proof engineering. So, I don't know if you saw it, the repository for the Navier-Stokes proof that OpenAI put out. None of us can understand it. Uh it's a They use a proof checker called Lean.
26:27And they open sourced it. Kudos to them. And then you can go and see all of the proofs that this thing wrote. It's basically like millions and millions of lines of mathematical proofs, but they work because we have a compiler, we have a proof checker called Lean, and so we know that it's mathematically sound.
26:44Very few people in the world can understand what it even means, but it works, right? And so this is This is a risk, I think. On one hand, AI can create probably correct systems. It could be doing things that we don't fully understand. It could be arriving to results that we don't fully understand. And so, some of that is scary. But again, we can put guardrails around them and and we must.
27:09So, you can build it into the software factory to say people can go faster, but those things will be built in to Maybe to make it super practical for the software factory, you can put guardrails around performance.
27:19You can tell the agents, "Look, this critical transaction, maybe for Salesforce their MCP query that says, look up customer by name. I want to have a P99 of 200 milliseconds." The agent will, you know, go to whatever extent it needs to go to satisfy 200 milliseconds. It's amazing to watch.
27:42Now, the same happens with cyber attacks. In our in our article about everything hackable will get hacked, we started reading the the reasoning traces of this open-weight model called Kimi. And when our CTO started reading what the model was thinking, he frankly got a little bit scared. Cuz when you give the agent the task of find a vulnerability, it's so resourceful.
28:06It's so unrelenting.
28:08You know, human hackers get tired and they need to get paid and they need to eat. These agents just you give them an an objective and they go go go go go. So, it's going to be a positive and it can also be a negative.
28:20All right, my last question. Um So, when I was working on with in the Slack unit, one of our best developers who just totally but I was actually through his child, it's kind of funny. So, Claude 46 came out February, I think, like kind of hit that tipping point. And he had tried previous models, versions and and he's a very very good engineer, wasn't that impressed. And then, um he was working with his daughter and he said, "Well, let's build something."
28:50And and he thinks he's using Claude 4 6. All of a sudden, he's like, "Wow, this the world just changed." It just blew his mind. He's one of our most productive engineers, and he doesn't just use one uh agent. You know, he has an agent, you know, doing the plan, doing the spec, and another one debating. He's got one writing the code. You know, the that whole An ensemble of agents.
29:14It's like an orchestration of agents.
29:16You know, in a negative way, the Hugging Face incident was a, you know, uh community of agents. So, I think they called it.
29:23Um, working together. He, in a very small way, has a community of agents, but he's kind of doing it in a bespoke way. If I take kind of the middle developer in our stack rank of really good developers, they're producing not as good code. It's not He's not as fast, or he or she, I'm just using it as an example. And hasn't really learned the pattern uh that the best developer has.
29:50How do you see the future where we can instead of like saying, "Gee, you need to like do it like like this guy."
29:58How do you bake it in so that everyone can go as fast as our top developer?
30:02Yeah, for sure. So, I I also have arrived at the same conclusion, by the way, that the best agentic engineers are harnessing multiple models. So, um, It's not just multiple models. It's multiple agents.
30:18And multiple roles within them, absolutely. So, what And by the way, this will save you costs. You'll be really happy to hear, right? So, with AI Gateway, one of the uh we offer developers, you know, one API key, multiple models, and one thing that we've been noticing is that the open weight models in in aggregate token use are now towering uh over uh how much the proprietary models are used.
30:44And so, when we're designing these pipelines for engineering, we have to think about using models and using agents much like systems, right? This is why I think the factory analogy is is is cool to think about because you're designing this sort of like production line.
31:01And so, you have at at the at the beginning of an agent that sort of creates the blueprint. And you might use a very, very smart model for planning. And then that hands it off to another part of the factory that swarms out to implementation.
31:18And then every factory has a quality assurance step, right? And you have a different agent in a different role using different tools and being given different instructions. So, it's mesmerizing to watch agents these days do QA by taking screenshots, looking at them, and saying, "Yeah, this looks good."
31:37Or running benchmarks.
31:40Yeah. And it's crazy the AI is looking at the production line.
31:43And it looks at the result. It looks good. It reflects to itself. And so, going back to the what what what what do we bring to the table? Well, I think it's designing these pipelines. I think the the best practices of engineering today are what exactly what that engineer is doing. It's like thinking one layer above, "How do I give my job to the AIs?" Uh and the other one that I that I think is really relevant to this audience is at the end of the day, we have to make customers happy.
32:11We're we're all working in service of customers. So, infusing this factory with your context of, you know, you had you had a call with a client and they give you a bunch of feedback. Uh they're filing support tickets. Can you can you automate the can you feed that into your factory so that it's not up to uh even a really good engineer might not just have the context of what to build.
32:35And so, helping them prioritize, I think automating the software development life cycle we call it the agentic software development life cycle is is is the future of software.
32:45All right everyone please thank Germán Roach CEO of Vercel founder. I hope you enjoyed this enjoy the the rest of Dreamforce we got a lot more coming at you but thank you for being here today.