0:00DHH did the funeral last week and he's right.
0:04Yesterday's slop is today's content.
0:06I am burning more virtual money than I'm earning.
0:10The society at large does not want an assistant, folks.
0:14I'm pretty sure society will just burn the data centers down.
0:18Just harshing everybody's vibe right out of the gates. Hey everybody, welcome to episode 11 of State of Agent Coding.
0:35We don't we It's a podcast. My name is Ben Vinegar.
0:39I am a co-founder of a startup called Modem. We do uh I don't know, we try to inject customer feedback aggregate into your agents. Armen, what's uh what's your thing?
0:49Hi, my name is Armen. I'm a I think it's called MCPO at Arendel. We built right now coding harnesses and interesting other things with AI. Um and I think the main project that people probably know about right now is Pi. Um yeah, that's me. If anything else happened in your past, no one cares.
1:13Okay. Um All right. So, we're joined here by Jeff Huntley. That that's your name. But who are you?
1:22Hey, I'm Jeff. Um, people might know me from like I guess the beginning of the year in the last year for doing some very unreasonable things with LLMs and kind of like the RA loop and the the kickstarting everyone doing loop engineering and all the rest. Um, dayto-day I'm a field CTO at a company called Antifysus.
1:44We do um deterministic system testing and property based testing and essentially we work to um validate and have verification of what your agents do.
1:59And um astute viewers may notice that Jeff and I are occupying the same physical space.
2:07Um you're here in Toronto and I think you got off the plane like yesterday. I sure did. It was like 22 hours of flight. I'm here for here for two days and I'm back to Sydney, Australia. So, just visiting here loses me almost three full calendar days in transit and time zones and I'm lazy and I didn't move. So, I'm in Vienna.
2:34You know, one of these days I'll get to Vienna preferably when it's uh when like it's not like flooding. I've seen what happens when the snow melts there.
2:45I really love this idea of like schools of thought, like AI schools of thought, but there's actually physically a school and you walk the grounds and like um you know, as you could see, here's our slop, you know, our slop students are are learning the Vienna style over here on our right. This is like he's he's showing you the grounds.
3:03So one one random like Toronto thing really quickly is so this is I don't know if are you familiar with Drake in Australia unfortunately.
3:10No I think someone did a study once that Drake has contributed contributed to the GDP of Toronto in like you know tens of millions maybe like a hundred million dollars just sort of his you know visibility on the city. Now, where I'm going with this is what do you think is your is your contribution to the GDP, the collective GDP of AI development from Ralph Loops?
3:35Oh jeez. Uh I had uh the open router folks personally come to me and say thank you very much for the uptick. So I know for sure it's increased their valuation through to people actually burning more tokens.
3:49I mean basically they should all be funding you is kind of the the thing here.
3:53Sure. I'm gonna say trillions for shock for shock value, but it's almost like plausible. It's actually borderline plausible.
4:01It It's kind of weird. Um you published the research back in July last year and it took like Silicon Valley 9 months to implement it. I remember when I first showed it to Kurser and they they thought it was garbage. Next thing you know, they implemented it into it 9 months later. Um, and then Anthropic did it and then OpenAI did it and then we now got goals and everyone's talking about loop engineering. It's it's really weird, but like the net result of it was
4:29software development. The between engineering and development is I think it was like $1042 an hour Dex calculator when you run in the loop and it builds AFK overnight.
4:42That's kind of weird. And that actually weighed a lot of my shoulders when it first came out. like ah you mean like so I I did a bit of uh travel this year a lot of conference talks around the world just pleading people to actually invest in themselves um we'll get into the pdoom stuff later but my shortest take on that is people are failing a curiosity test right now if they're a software developer software
5:11engineer I mean if if you're displaced through AI it's just literally self-inflicted because you haven't been curious right now in a time of rapid change. The only reason like we get into the PDoom stuff later, but my my honest take here is um if you're displaced because of AI and you're a software developer, it's because it's self-inflicted. You haven't been curious in a time of rapid transformation.
5:35You trade time and skill for money if you're an employee. and uh employers tastes are changing rapidly, but there's no reason why you can't keep up to date with what's going on. Just harshing everybody's vibe right out of the gate.
5:53Yeah. Oh, I had so many conversations over like the last three months which were like sort of exactly and this is like you encourage people to try things and doesn't necessarily vibe. I would say [laughter] I think if like people don't want it then they don't want it. Like can't really change it. You just got to wait.
6:13Let's just do it. where [laughter] I I think on this topic I'm very I'm very sensitive to this challenge for a lot of people because I think that when a lot of people come into the industry um you know you're young and you have infinite time on your hands and now a lot of those people are in their 30s or 40s or more and they do not have the disposable time they do not have you know they've racked up a lot of dependencies
6:41you pick up dependencies as uh as you age, okay? And I don't necessarily mean dependence. I mean, you know, your responsibilities in the world start to rack up. It's the same reason why, you know, you get old and video games are hard to get stay into. It's it's not cuz you don't like them. Maybe it's just, you know, between everything else, it's hard. And I think for people in that place, I just think it's hard to retool and with with the rate of change. Today there was a tweet from DHH where he said
7:09something like um like I don't know exactly what he said but basically like you really should be optimistic in life it's easier to live life and then the responses where a lot of people like it's easy to say if blah blah blah blah and my situation doesn't afford for it and I think like the the reality is that for a lot of people it's very hard to feel optimism if the thing that they have like has been at the be at the center of their like income come stream and and life and like selfworth
7:38and everything feels like so much in flux. It's a good moment to sort of adjust and be flexible and do all of this, but like it's easy for us to do. I would say I think I understand it's not easy. It's not easy for me. I don't know about you guys.
7:53Well, I I concur. For the last kind of almost two years now in my writing, it's been described as like the depression era of Jeff where I'm mourning the profession and then pleading people to be curious um and to just invest in themselves and put themselves and their family unit first. It is hard. This this change is forced upon the world non-consensually.
8:18It it it displaces your identity. Are you a net developer? Are you a Golang developer? Are you a Python developer?
8:26Doesn't matter anymore. It's just how good of a developer are you? And are you using the thing that employers now expect as the bare minimum?
8:36Yeah, that's a lot of burden to carry on your shoulders.
8:39One of the most common tropes in in AI anything right now is that people talk a very confident game externally, but then themselves are using that as a way to talk themselves up. I think this is this probably also to some degree applies to us.
8:56Oh, like we we all talk a big game, but we all go home and suffer in silence. Is this what you mean?
9:01Well, I mean I'm not I don't I'm going say like suffer in silence, but it's like feel a lot less confidence uh than we s maybe portray to the outside world.
9:10Oh, I'm straight up about this. I don't bloody know. Like every time I discover something new, it actually strips me raw to the point where like I I don't know if I don't know. And that's really weird at my my position or even your position to say you don't know. It's a really good point of just how raw it strips you because you discover new technique, new capability or new trick type thing.
9:38You're like, I thought I had a mental lens on this thing like how it's all going to play out. And then the game changes randomly since you're here in Toronto and I just love to play tour guide, but there's a relevant thing. Um, the closest US cities to Toronto right now are Buffalo and Detroit. Buffalo is an interesting one that I think even a lot of American citizens don't understand.
10:05It's its piece and history. Buffalo was the third most populous city in the US.
10:12It was incredibly prosperous. And the reason was it had rail access to the Great Lakes. It has been one of the most declining I think it is one of the rapidly most rapidly declining uh cities in America. I think it was like 600,000 people at one point and today it might be like 250,000 you know Armen has this open. And then of course Detroit another but later and similar story where of course they had the auto industry you know technology or competition changes and that place got
10:40kind of hollowed out.
10:43I remember living in San Francisco and I would tell people even before the the um co just like you know if there's one thing I've learned just from history here in the US is that there you there's more there's a bunch of cases of people with hubris of believing that the great times will never end that this you know this thing that we have in this city is forever and then it's gone like people can't even can't even uh fathom that
11:10it's going to change for them Right? And everybody rejected this to be clear. Like nobody even humored me on this idea now. Obviously AI is still software, right? So I'm not trying to say that we're devastated, but kind of it's like a it is like this.
11:25It is this it is changing. Um I've been to Detroit. It's not just automobiles, it's also music. Yeah.
11:33Yeah. Um, I would say that as someone who's been in the game a long time now, I've been through a few of these cycles from the original.com bust, uh, boom and bust. Um, the reason that things got so ripping like hot um, during the ZER era, uh, for someone who was senior where it comes to topics of compensation is because in the in the bus cycle, we didn't raise enough genius. So it
12:03created a scarcity of knowledge. So thus uh the comp went through the roof. Hold on folks, the same cycle is going to repeat. I swear the same cycle is going to repeat. CS uh is down. Um I remember something like 22% in Singapore this year of enrollments.
12:24Um you mean the same cycle is going to play out. If you really love computers, you should be all in on loving computers right now. It everyone's having the time of their damn life. And for people who have been curious and can on a whiteboard explain how to build pi code like like a sequence diagram, how inferencing works, what what a prompt ca prompt caches, how that infill works, all the rest like the basic curiosity
12:53stuff and kept the skills up to date. You can get placed anywhere right now.
12:58you and I know I know a company in Australia that will place someone at 1.5 mil AU um just for being really good at this stuff. What are we doing here? You I mean well I think like there like two things that I sort of like a little bit competing here. One just like just because Detroit did not grow as a city after certain point doesn't mean like automobiles or automoils stopped being a thing. And
13:27what actually happened like in the Detroit situation was I think very much like a combination of um that technology turned from well Ford was building all the cars to there was a lot of worldwide competition and everybody wanted to have a car and there were many car companies and I thought the Buffalo one fit more to be clear. No, but I think like it is it is actually kind of relevant to to like what's happening currently because like at one point um and this is the point where we still
13:54are is that our industry got paid like you as a software engineer got paid really really well because it was a scarce we were scarce resources in that market and um and our competition was just like other software engineers and um and our industry did not commoditize and like automation did not play a role.
14:19Um and that is definitely changing. It doesn't mean that there is no future for software engineer but um like I think our exceptional role that we had with our industry and our profession um in sort of the last 20 years probably will not come back in the same form. I'm I'm absolutely convinced of that. Um but it doesn't mean that it's not still interesting but probably realistically speaking there will be for some period of time there will be more software engineers because there will be a huge
14:49demand for it but I think that will commoditize really quickly. Um I think the question on San Francisco in particular is interesting because like we're about to see the largest IPO ever for the city with an humongous amount of wealth being put onto San Francisco. a city that has not really managed to grow its housing stock. So like I am actually really curious what's going to happen once once the Tropic IPOs because there will be a lot of melted
15:17multi-millionaires um competing for for that city. I think it will be interesting to see what will be the last last big horra I think in some form or another. [clears throat] I'm hoping we'll get another like PayPal mafia type thing. Um, you I mean like you have a few artists in the art industry or in the media industry. They do they they do a good movie, make a bit of money, and next thing you know, they want to start writing seed checks for
15:47the thing that they already know, whether that be in the media or arts.
15:51Well, it's also same in tech. When people are minted, they like to invest in what they know and it allow creation of what they know. So, I'm hoping we we get a bit of a renaissance to your uh point. Yeah. Uh right. Right now, like the demand for people who know how to build pi code by hand on a whiteboard is through the roof.
16:16Um if you don't know to how how to how to do that, then fix that. Have some project showing equivalent of that. Put that on your CV and your acceptance rates for interviews will go be much better. folks.
16:29Now, I could see it like commoditizing down again in a cycle, but that's to the cycle of the future. The game has fundamentally changed. If you're in love with typing on a keyboard in an IDE, I'm sorry to say, but DHH did the funeral last week and he's right. But it's like the era of typing code. You can still do it, but you do it in your personal workshop.
16:58Like employers now want you to use the CNC machine and be very good with the CNC machine. They don't want they don't want anteal woodworking anymore.
17:08Jeff's talking about Rails World keynote from David Hannah Hansen, DHH. Hopefully I didn't butcher that.
17:17Let me let me summarize what happened.
17:18Someone was tweeting on Twitter just as Rails the Rails keynote was starting is that uh sort of like an outrage tweet that DHH was um talking about everything other than Rails at the Rails conference or Rails world I guess it's called. Um, and so he was on stage making an anal analogy of how over like three generations of his
17:47family pictures became both photos from like a drawing from painting to a photo to like smartphone pictures of which you have plenty. And like first you had to have a painter and then you had at least a photographer and now like whatever there's like so many more pictures than they ever had. like he basically had one drawing I think from his grandfather and then a handful of pictures from his childhood and now his kids have like every second of the life photographed.
18:15I think he I think he actually made a really like I actually thought it was a really interesting story because he was basically saying that part of the art that you have appreciated or you you understand exists is actually um an economic outcome, right? The first portrait painters cost a lot of money.
18:31That's how you actually got your family's picture on the wall. And as the photo as photography came about, that business kind of went away, which was to say, you know, drawing reality isn't economically viable anymore. And that led to impressionism or people drawing, you know, um, you know, uh, you know, getting into cubic periods or or strange sort of like, well, let's draw what's not reality. And I also want to say like on a very personal level, one of the reasons why I found this very interesting is I was at Rubicon for
19:00Austria earlier this year where I was on on a panel with Chad Fowler who is also one of I think the more influential people in the Ruby world who at that point in time already said like look the agent never writes Ruby for me anymore.
19:14Um and so it's like when when you sort of sample out what programming languages play a role, Ruby really doesn't like meaningfully show up. And I was already curious like what the what like what like as I see the HH do more and more of of programming like how was he respond to it and he actually did respond to this in a in a many ways not surprising way which is like telling people that the world is changing. Um but he I think was also pretty correct in his assessment of the situation. And I was
19:43very curious to see how this lands because um one of the responses I think um that was very noticeable online was a complete dismissal of the message.
19:55Um but I actually think that the inperson version of this was actually much better received. Like I've I've paid attention to one of the twe uh one of the threads on our rails over at the subreddit is called where someone who was at the conference said like hey the the mood is actually way better than you would think. So I think it's really interesting because like these communities are rapidly disintegrating and reforming around ideas that they like made them into that community in the first place even if they're
20:24completely dropping or changing the technology. Um, so I I wanted to talk about this quite a bit because like in some way the Rails community is actually a descendant of the way DHH saw software development to work. He just chose to use Ruby as a tool.
20:43It was a beautiful uh UX that it it looked nice to his eyes and it was good for human to write. But what what is what is the point of a programming language with sugar or a framework with sugar designed for humans if humans aren't writing the code anymore? So there's a really good um been chatting with the creative uh elixir in the last week and he he gets it. The languages
21:10need to evolve. Instead of doing like say like operator chaining or what else have you like in designing like sugar for humans like all these programming languages have the the designers and authors have all designed it as if the human operator is driving it and they've dumbed down the language to make it accessible and easy to be learn. There's all these things that are now falsified now that the agent is is writing it. So like what does the agent need to to be
21:39able to write uh to write code in that programming language like LSPs fundamentally what's the point of them anymore? But does that mean that the we're going to get some new type of thing for hooking the lang the language itself like [snorts] thinking like an agent running GDB type topics in on a per language basis like some some new forms of interpretability uh and it it's going to come down to
22:07whichever programming language community and authors really understand that this paradigm has changed. It has changed. Doesn't mean the language is going away, but those that swallow this tough pill and evolve will be the ones that survive. And I'm very thankful for DHH delivering that talk. I will say that it's quite shocking to hear, but developers speak with other developers.
22:40It's kind of like a couple of cows in a field going moo moo to each other and uh completely missing the point that eventually the cows head to the abattoire. I know that's kind of grim but like developers stop being cows and actually learn a bit about about business. Like speak with founders, speak with investors, get curious. Like learn what's changing in this climate. Don't be a cow that speaks to another cow.
23:06Like learn to be a product engineer. learn how things have changed, how the expectations are changing. If you're working in a company that has banned AI outright, you should quit that company. You're heading to the abattoire.
23:19Are there any of those left?
23:21Yeah, they are. [laughter] I mean I think like one of the one of the ways in which um I'm noticing that the message does land in one form another is um many of the early holdouts on aentic software engineering have like really turned around and I I remember like even DHH himself was quite dismissive of the utility. Oh yeah, he's so
23:50early on and then I remember Notch was very dismissive on it. Notch turned recently. An armac people carax this stuff's pretty legit nine months ago and people out there still today saying it produces crap code and meanwhile you got liners using it.
24:08You Yeah, but the thing is like it is it is it like for me the strongest signal that like actually something is tremendously changing is not so much that I believe it's changing. it's that the the people that are very vocal against it who have a lot of experience are slowly turning or or quickly, right? And and there are not that many that are like really established and a complete holdouts. I think we're also going to see uh that take off. Um,
24:36what I also remember from the keynote was, and I think part of what made it relatable is, you know, DHH actually being very transparent about the fact that he's turned around on this. And I think he said even a few months ago, we we thought, hey, let's just do all of our design with AI and oops, actually, you know, that wasn't so great. So, we we pulled it back. So it sounded like he had expressed sort of like experimentation but not feeling convinced but now he's sort of
25:05completely on the other side and I felt it was more like come join me like come to the other side with me.
25:11Exactly. He did a fantastic job of uh a a tough message that need to be said but he said it very well.
25:20Yeah I think so. But I will also say that I think like um there's a very high chance that because it was DHH who said it, it will not land because he's a very polarizing figure.
25:33Um, and and I think like while I do think it's going to have quite a bit of an impact on on a whole new community or at least on the Rails community, um, there are like much larger holdouts where I think it will take quite a bit longer um for someone to turn around.
25:52So, but what can you do about it? Um, I guess I can almost segment people based on the social media they use for the last 2 years. If you've been on X, um, you've completely ahead of the game, you know what's going on because all the AI tech things has happened on X. Even though I I don't quite like the X platform and that's not on political purposes. It's just like the amount of AI reply bots right now is through the roof and it's not great.
26:22Yeah. But I think like this is this is important in a way because like the the unfortunate reality is that um a lot of the platforms where all of this is happening right now are in for one form or another sort of very politically leaning and I'm also not a fan of this. Um and I think it pushes away a lot of people that would otherwise actually be much more willing to engage in some of this. But it's like the environment is
26:50kind of poisoned and and I I think people really need to try to look through this a little bit or look past this a little bit.
26:59They do like like like if you want the alpha, you have to push through and like find find it on X. Now for the last two years, it's all been there. People are posting their thoughts. We saw the creation of OpenCaw before it was open claw pie in the early days. Yeah, not on LinkedIn. And I tell you what, if you go to the fediverse and talk about AI, you'll get your entire like Mastadon
27:26instance banned. And uh if you go to Blue Sky, yeah, there are some high alpha accounts there. But unfortunately, the tech community has fractured on political lines and and factions, which means a lot of this this isn't getting out. Um but what what can you do? like uh personally I've I've taken a lot of time off work unpaid this year to give talks about these topics.
27:54Um DHS has given his talk. The individual I know at least one programming language author is like yep we need to change the languages because agents are running the code now not humans. So we need to optimize for the agent not the human. And so it will have its ripples but like what else can you do like you try if you try getting I remember when I was on hacking news my early writings when I was saying hey is anyone seeing this it's all changing IDs are dead. Everyone was calling me crazy.
28:22This is back in like December 2024. I'm saying are dead as a product. Everyone's calling me crazy and I was just writing hoping that other people would pick up the phone and notice like my my peers looked up. I remember like PC like Fly.io. I managed to nerd snipe him and he's like oo uh I think I can now use this to automate the creation of my sap rules for security. And like I started
28:51collecting all those people or like like the peers slowly and you start seeing these attitudes change but like you mean who should be carrying that that that burden and water.
29:04Um I'm I'm past it now. I've I've written enough. I've dedicated enough personal time and like with the loops coming it was heavy on my shoulders publishing that research. But you you you can't you can't just carry that water all yourself. The best you can do is uh if you know this to be real, if someone gets curious that you know, you need to get them to their own moment. You like find something that
29:32just makes their mind explode and then ask them to show someone else the same thing to that person. But it's the problem is it's highly personalized.
29:42uh for me was seeing how easy it was to build like pi code or a coding agent and how you're building your product recursively and I was like oh this is like nothing you ever seen before recursive product development it's like wa and as soon as you you you taste and you feel it you realize everything's different but like you and if you're not building a coding agent as a business how do you get someone to understand that
30:10here to come up with insane analogy for this.
30:15Has anyone seen the um Bob Dylan biopic?
30:17I think it's called Nowhere Boy. It's the recent one with Shalamé.
30:22I watched this basically not very knowing anything, but it was a pretty good movie because it takes place uh not over his whole career, but over a sequence, which is kind of the beginning, and then his transition to rock music. And so the beginning is like he's a folk musician and the folk, you know, music industry is like, "Wow, what a talent." They put him on a pedestal.
30:42He is growing their industry. They put him on their festivals and he's, you know, everybody's happy, but he wants to keep experimenting and he starts getting into rock and other music. And now he comes back to their big festival. This is like, this is now the climax of the movie and he's like, you know, he's invited back because at this point he's royalty and they're like, but you're not going to go up and play your rock music, right? Right. So, this is his like rebellious moment where he goes up onto the stage and he's almost trying to
31:10there's like now there's younger people.
31:12Basically, it's a generational change where there's younger people in the audience. They want to see the new Bob Dylan stuff, but you have the sort of old caretakers who are literally I think even getting the police to like shut down this thing. And I think it's fascinating because it's like, well, folk music literally was one of the most popular, you know, genres of music. Of course, we never think of that today because it's no longer true. rock music then took off and he rode that wave and his career, you know, the reason that
31:40you know who he is today and not say many names from the folk industry is because he evolved, right? Anyways, and now I think even rock has actually declined pretty hard and it's sort of like now that you know there's another thing and I I guess I just look at these parallels where boy we don't like change.
31:58No, we don't. I think it's al like I think like one of one of the parts where like it's not just a change that like like society goes through. I think it's also like everybody's getting older and as they're getting older they're changing. The Australian version of this I think for me is Tim mention like he was a comedian comedian musician for like all his career and his last two albums are like the darkest stuff you could have ever listened to. And I think it's just because like his he got older.
32:25Um, and and I think like this is the like in in some form what it was a little bit weird about our industry right now is I feel like we were all sort of similar age and sort of like aged through [laughter] and now we also see the world different because we're like older, we have kids, we have families and we kind of forget about the people that were like 20 years earlier and went through like the com bubble and then got kids around that time. Um, so it is like it all sort of goes together a little bit.
32:51I can give you some cool things. So, and I I one of the first things I wrote was uh dear juniors, you're screwed if you don't pay attention. And uh it got syndicated by the primogen. Thank you, primogen.
33:09And it it landed into that demographic of juniors. And he read it, he narrated it, and he agreed with some takes, disagreed with whatever. But the net result was a lot of juniors read that blog post. And I wrote it as if it was like those 20somes were my son who was uh like 20 if he was 20 like career advice what to do explaining what's about to happen what you should do explain the cycles which I mentioned previously of like the boom and bust and how it all works. The cool thing with
33:38that is I've actually had people when I've been to the AI engineer world fair and like when in my travels they come to to me and they're just like thank you like I I've got a promotion just because I was like I'm I'm new but I got promotion because I came with the attitude and I had the skills because I started to invest in myself like dude don't thank me you did all the work you mean you were curious you invested in yourself somehow the the my writing came across your
34:08Maybe it's by the Primagen. Thanks, Primagen. The end result is uh I I my LinkedIn inbox is drowning with these stories. I've got that is in person. I get people asking to take selfies. That is weird, folks. I'll take a selfie. But like it is weird. In Korea, someone asked me to um they wanted my autograph uh for for for like a canvas painting uh to hang on the wall. Apparently, it's a thing in Korea.
34:37That also might be a bit of uh identity fraud. We'll see. I used a different signature. Um but it's it's weird. It is weird. We'll probably tap this this topic out. You know, this week Armon uh you ship some stuff.
34:55We shipped some stuff. Yeah. I mean, you mentioned earlier like not everybody is a coding agent developer. Um we sort of are now. I really don't want to pitch pi here, but we did build something. Uh, so we shipped PI one, but we also built something that um actually took us, weirdly enough, so it's I think it's around 10,000 lines of code that we shipped, but it really took us four months to build.
35:20Four months in 2026. What is that like a fiveyear project?
35:24Because these this is the this is the part L and suck at the the shape of a contract and the feel where it's human.
35:32You can express an idea really fast but like getting the right feel requires a lot of knowledge and a lot of experimentation. The weird thing is like we have solved this problem like I feel like 20 times within two hours each and like not felt like it is actually a good solution because like the agent solves every problem you ever had in record time but it doesn't actually make for like a system that's inherently like
36:02functional and has the right trade-offs and everything. So like finding the right trade-offs took us that long. Um and um and what we ship in the end is is this thing called pyurable which is basically how we now think that an agent should be structured. So it's it's not yet used in the pi coding agent because would break quite a few extensions. It was just integrated like um like we want people to use it. Um but it is um it is a
36:31different way in which you build an agent where like the agent orchestrates in itself. Um like the harness like the basic idea is like you shut down the harness and you restart it and it keeps working exactly where it left off and it supports it over network and it supports that over like really complex interactions where you for instance spawn sub agents and they want to communicate and it can turn off the whole thing and can bring it back up.
36:54But that's not really that interesting.
36:56But I think what is a little bit interesting about it is that first of all it actually took us that long to build it which in itself I think is hilarious because I've heard engineering is solved. Um but the other thing is also that I remember last year pretty much exactly like 14 15 months ago um we had this very strong conviction I think at least Mario I Peter and and a bunch of others that we talked to that file systems and bash is the way to go. like we we were
37:25like very very convinced like that that's like that's how you should do it but we we didn't actually really know how to build an agent harness around it and so we basically copy pasted what cloud code was doing and cloud code has a very simplistic idea of what a session is which is there's a transcript you you it's like hey please do this and the align gives the tool call and then you give the tool result back and you keep doing that in a loop and yes you and continue the
37:54session in one form or another. But um but but that sort of very simplistic model has has has stayed at the core of every single agent more or less up to this point.
38:06It has um the the most simplest thing is it just is essentially it's all it's done is automated copy and pasting.
38:15But there are a bunch of things that have actually happened in the last year where all of the sudden that model no longer works. And I just want to give you some ideas. So one of its for instance that if you want to compact in the background all of a sudden you do two things simultaneously. You're compacting one thing but you're already making progress on another thing or for instance you want to have a Slack experience where five people simultaneously talk to the same agent and some of them might talk to the same sub thread other ones the agent might want to branch off. So we're
38:44we're now living in a world where we have much higher demands for what these systems are doing. um and it mapped incredibly poorly on um Pi's original design um and so that really led to it and I think it is for me at least I think it was an interesting moment to reflect on that is a little bit of cargo culting going on and I think like we all we're all sort of suffering from this a little bit where um we kind of like build more
39:12and more stuff on this very basic primitive that everybody's using but actually turns out to create so much pain and suffering on an engineering side everywhere. Like, yeah, we all have the same problems, but like we don't notice this problem so much because like you can just as an agent to build it and it will just build you a decently acceptable agent, but if you were to take a peek under the hood of like how it works, it's super messy and and like
39:40horrifying and totally not reliable. So yeah, I don't know really what the point of this here is, but I I do want to say like actually taking the time and effort every once in a while to look at like is it really the right way of solving this problem is is very much still a thing we all should be doing. Um because like this technology is born really really quickly and is in many ways held back by ideas that we had about how it should work um that are like a year plus old.
40:09Can I get this? Can I get Can I get that louder and printed on a shirt? Because I completely agree. Like, so here's my hot take. Um, your agents MD and a lot of your skills that you use should be deleted after every model release, you know, I mean, or like every couple of months, new model release and redo some of that stuff. There's so much that it we carry and we think of best practice even today in software engineering. And this is the way that we need to do things and
40:39they're going to perpetuate that line of thinking um going forward. Like we need to actually think about what can we use to delete or what we how can we use this to redesign something that we already have. Like I think um and some of the advancements in the last year is like okay we had this a notion of like a steering
41:04message. The idea is that you could normally when you were in that chat loop, you could type a message and then there was always that question. Okay, it's inferencing that message and now you got a follow-up message. Do you need to cancel that inferencing request and append to ste to steer or is it a follow-up type thing? And that was all a UX paradigm. Nothing really changed under under the hood. It was was more how we drove the harness, but the actual loop itself stayed the same. And then we
41:34started to get um async tasks um for like bash and all these other tools. That was one of the last big advancements. But like we do need to actually continually think about what do we need to change? And it's not just about this. It's it's wider. Like think about why do we even have like a like a like bash or a shell terminal, you know? I mean, you start thinking
42:02back long enough and hard enough, the reason that we had a shell terminal is because we used to have a human operator. You know what I mean? Like, if we never had that need for a human operator, would we actually have an operating shell? Um, that's the level of things that what I think to is how we can redesign substrate in the application.
42:23It's not in your operating system. So, there are like we we do have to really think think about things how things are designed differently. Can I ask just to make this even concrete, what are some things that you guys have thrown out say in the past, I'm going to say three months, we can say two months, you know, since you're talking about deleting things, what are some, you know, deleting your skills? That's good.
42:46Are there other things that should be thrown out or maybe not should be but are worth exploring throwing out?
42:55I've got something that I published this morning I can show.
42:58You're going to throw that out already?
42:59Yeah, throw. Look, it's a it's a concept. It's not that I'm going to throw out, but it leads to a line of thinking or throw out. We've raised two generations of software developers who probably have never seen Lisp before in their life and seen what it can do. I'm going to show you what happens when you hook Lisp to an LLM. Um, I'm very curious because I just even like three months ago I gave a talk where I said like lisp is the worst programming language for agents because
43:28they're terrible at counting closing parenthesis. That is the old joke. Now I'm not advocating maximal for lisp but I want to go this from the lens of what can be thrown out. So people are used to the idea that the back uh like a coding tool a coding tool would you you got to you got to you prompt into the coding tool.
43:52It runs uh like a bash tool to look to look at your project directory to figure out the file name to edit and then it uses the edit tool to add the functionality that you're prompted. This is their mode of model of thinking of how it all works. And then of course you got to do the compilation step and then you got to do the CI/CD and you got to do the code review and you got to do all these things. What I'm going to show throws that all out. Uh it is very
44:20unhinged. Um but anyone's used to lisp should be used to some of these ideas. Yeah. And the people looking to do software factories and that's the new hot buzzword whatever. It's really process automation, folks.
44:34All right, let's let's see this. This is a lot of hype. We're not Are we hype merchants? I don't think so.
44:43So, what we're looking at here is something very odd, very weird. Um, the typical way that software is developed now with agents is that it edits a file and then it does a compilation and maybe that triggers a hot reload and it reloads your website, your application. type thing. And that's the way agents develop software. I'm going to show you the inversion of that. What I've done is connect an LLM to lisp.
45:14So I can prompt it for things. I can go what I can go what functions are defined. And right now the lm is asking my running lisp environment what functions are installed. Let's do something fun. Let's do something fun.
45:37Let's do some programming. [laughter] Hey, I want to be able to reverse a string. Can you define me a string reverse function? It's even got typos here because the lms doesn't give a It just does it. So here right now is I'm chatting with the LM and it's defining a function into lisp and I can
46:04terminate this lisp application and I can go what functions I defined and I'm iteratively programming my programming language through a chat with the LM and once that that function is defined it's defined mind and I go okay I now need to make a string
46:30uppercase give me a function and this is very different this is not your typical uh edit file compile whatever this is me chatting with LM and it's defining the functions as I go I know that the open code guys just had a blog post very recently that was like here is all the work that we had to do to make it so that the individual plugins in our system could actually hot
46:59reload. Never mind real time program them. Although maybe that's possible, but even just to frankly have a running system and have one piece of it refresh was like a huge architectural, you know, challenge.
47:13But we're talking about something completely different which is ultimately Yeah. How is the words?
47:18It is weird. It's code is data. And um I can go undo this change. I don't want this function.
47:28And next thing you know, I'm going to roll back. So what the roll back to the previous generation. What I'm actually doing is I'm actually editing the the actual the actual lisp image and the runtime environment via this LLM. And it has two tool calls defined into the inferencing loop. Armen. one is um like either execute expression or define expression expression as a function.
47:58And in this case I can now go yo run I now go yo run the uh reverse function and go hello Armon. This this is essentially going to inspect in the world what functions are defined or what what's available for the programming language and then it's then going to execute the
48:26expression the same way you stand the standard tool call. So I can just execute this I can just like execute this expressions like just as as like a notebook here ripple lisp style or I could ask the LLM to do it and once it's defined it's now defined in there and then I can like take this up. Yo I want the ability to create a to fetch a URL.
48:58um the fetch URL and then pass the JSON add it please you know you can keep going this all this way this up and this is kind of like really strange because once this is done it's done and I want you to think about this not as a chat interface instead what happens if this is your programming development model as a user you logged on to your website and when you're logged onto your website you got some admin panel
49:28and then you want to make a change to the application. You're not loading up an IDE or editing source code or what have you. You pop a chat prompt. You got a little secret little hackers pie the net type menu that pops up that only shows for you or as a staff member. And then you can evolve the program that's running live by just chatting with. This is very different. And what happens if that's like a user feedback request?
49:57That was in many ways I think like on a on a single player experience was very much like what made Pi so interesting because like Pi can build it can rebuild itself and reload itself. The challenges obviously that you run into with JavaScript is um like it's not a language built for hot reloading. Um and and you do need to build the architecture in a way where it works as as good as possible. Um but but this idea of like software that extends itself I think is is generally very
50:27interesting because it it also allows you to create a little bit of um sanity in a in a world of like inference dependency.
50:36So is this completely your own unique runtime is this lisp? What version of lisp? What is lisp? Why do I care? Why should anyone care?
50:45Right. Okay. So, this is just something that I vibe coded up being old and knowing what Lisp can do cuz I want to be able to demonstrate to people there is something different to like this entire edit file compilation loop package deploy type thing like how can I like show people the idea that you could develop the software in the software and does lisp not have those things?
51:12Lisp has always been about these things, but no one that is aware of at the time had actually hooked an LLM up. You had to be a Lisp programmer to do lisp, but now I can just got this natural interface that allows us to do lisp type stuff. Um, and then I can define functions on lisp. Um, and here like I I've just asked said fetch me some example JSON reverse the value using the function and like I'm just defining Yeah. I think I'm more just trying to
51:41like help help people who maybe aren't familiar. So like you know hot reloading is built into it.
51:47Right. Okay. So it's like a key key thing is that the maj is the biggest thing like is this like is there a file are there no files? Is this an ethereal, you know, blob that exists in in RAM on your computer?
52:04H this I don't know how to distill it down. Um um and you might have a better way of explaining it.
52:11Maybe to make this a little bit more like concrete. Um the problem in many ways with software is effects. um and is is is changes that they're making which are non like I very like you can for instance like this is also why pi durable exists is like if you shut down pi and reload it doesn't come back exactly in the same way as it was before because not everything is serializable and and obviously like in you could like if you work for the
52:40company that uh job works for then um like it's all about like making the whole thing deterministic right like you can snapshot everything. But most software doesn't work this way. But lisp retains the ability to serialize its entire state out more than most languages do. Um, which I think is is is is quite relevant here. So like the the functions are sort of defined very similar to how data is defined in a language. Um, which is not the case in most languages.
53:10If it's so fantastic, why is this feel like you're you're unearthing like a you know a grave here?
53:17Why does it feel that way? like why did it go away?
53:20Well, for a start, Lisp is very slow unless you have a computer that really works really well. And so like Lisp did not make it very far um after a certain shift towards I guess the personal computer came where people wrote the assembly and everything. And a large reason I think why Lisp is horrible to work with is just it's a horrible language. It's terrible to look at.
53:42It's not a great language.
53:44It's awful for humans. But maybe it's great for LMS. Who knows? Um, I think like many of the trade-offs that we have done as as as as engineers um are really because we're lazy and we're basically picking the things that allow us to be lazy.
54:06Well, this is like the Ty Ruby thing. It's it's designed to make it easier and more beautiful for humans, but it's less powerful. Paul Graham has a great essay about blood programmers, uh, which basically goes into like explaining that some languages are designed for people not as high up on the power spectrum or as knowledgeable, etc., and they think it's perfectly fine. Um, and they target the blobs.
54:35Here is someone who is definitely not a blob. This is Kristoff. He is not credited for this, but he is the person that everyone's doing specdriven development. He published some research back in GPT 3 or 2.5 days. He's a engineer at GitHub.
54:58Next, it's one of the people behind F and Ionide. He's also one of the people uh that joined the VS helped build VS Code and now doing a lot of stuff with C-Pilot and what GitHub's working on next which is coming. He published some research on spec driven development um and said could some markdown be the future of how we publish software? Now, when I was doing all my raph loops, I
55:27cuz go back uh I remembered his riches like, "Hey, the models got better. Let's redo his research." Um, and the rest is history. And now everyone's talking about specs and like markdown as a spec. No, code is a spec, folks. Markdown is not a spec. Code is specification. But even in the last week, I've had some arguments that maybe formal methods are specifications, but like you know, and we've we've there's specifications and there's vibe specifications.
56:01Anyway, before we go there, I want to show you something. I would love to see this idea as a web app or a web page developed like this. Maybe I'll throw some agents at it. And I'm like, "Yeah, dude. This doesn't have to be a chat app. The entire idea is you should be able to build the product in the product with any like without going to some sort certain different sort of thing. You should be able to like kind of like rightclick pop Chrome dev tools and just edit the product in the product type
56:30thing and you see the end results of this. So this is what he built like vibed up in an hour. Here we go. He's vibed up a little a little web app which is this exact same idea. It's not necessarily using lisp. It's lisp is just the easy way for me to express the idea and also like to be straight up to nerd snipes and the nostalgia of people.
56:53Um and also like maybe teach the new generation that like there are powerful languages that's out there like lisp even if they're debx terrible but who cares about the debx now that agents write the code. And here he's just built a web app and instead of like going over to VS Code or PI code or what have you, he's uh he just clicks his little developer thing and he prompts in the functionality he needs and then it adds that functionality to the application and then it saves a generation. He's
57:22programmed the application by talking with it. I want you to think about what this means for someone who is not a software developer, a business person, someone in sales, someone in marketing, someone in finance or legal. Imagine if they could build your product just by chatting with the product and no one else is using all these different specialized tools like subcoding agent over here and then some SQL tool over there. What if you want to create a SQL
57:51query and all you do is pop this little menu here and you select all the different elements you want to create a report from.
57:58Done. And this is a just a completely different way of doing coding tool harness design because all of a sudden the programming language is enabled with an LLM instead of having um the coding harness produce some code uh for a programming language. So the you don't have to use list but not max will advocating for it. If you want to be a nerd go do this with small talk you'll end up on the front page of hacking news tomorrow guaranteed.
58:26I want to make a comment which is you ever do like an interview with like a film director or like a musician and they're like no come into my office.
58:34Let's let's take a musician right maybe someone like Beck and they'll they'll go into their office and they're just pulling records off the shelf. They have an incredible record collection from the past and what you learn is like a lot of their music ideas that they bring to the table are really just going through this sort of forgotten lore, right? Same with people hip-hop producers. Yeah. Just go a generation. Nobody remembers what this was. You, you know, somebody with that knowledge and I think the most um the most powerful people in music are are
59:02kind of people who grew up with that older generation music, you know, maybe they're in their 20s, they have this and now they can apply that. And what I what you're highlighting is like you're pulling out your old records of uh Small Talk and Lisp and um you know you're calling me old.
59:18Yeah. Old. [laughter] Well, it just sort of um these are some of the advantages you know having those in your um you know I don't know record collection. Sort of worth revisiting I guess is um you know it's worth going back to the classics and doing some sampling. I I think I would like to talk a little bit about um what's going on over the models because the models are changing and I think it relates a little bit to what um what we just discussed which is um we're moving
59:48beyond bash a little bit but not too much like the the number of harnesses which are now writing code to write tool calls is going up. Um I don't know if you notice that too.
1:00:04Yeah. Like uh the models aren't going for edit tool anymore. What they're doing is they're doing like a like almost like a program synthesis. So they they're creating these oneliner Python applications to do edit files. You notice that is this code mode?
1:00:18No, this is not this is not even code mode. This is just like the like there's an PI supplies an edit tool in like 80% of cases within Astra and Opus 5.5. It's no longer used. They're like you know what? Let's write some Python. Let's write a little function over there. And and it does this. But the reason this is happening, I think, is is a combination of two things. One is that there's very little punishment going on in the training runs of these models to like anything other than like they're
1:00:47succeeding. So like whatever means they have they will use. Um and and one of the natural results of this has been that the models are much more likely to use ad hoc code to do stuff. And you notice this in particular like if you give so PI for instance now gives uh as an option but we're probably going to make it a default uh code mode as a tool which is basically run some JavaScript in instead of running some bash and the difference is like where it runs. So bash runs where your code is like where
1:01:17your file system is and everything whereas the the code mode runs in a harness loop um which is typically the same computer but not necessarily so and that code behavior is trained into many of the models. So um Astra for instance and the 6.1 models are are excessively using code mode and the the weird behaviors you get out of those models once they start doing that is fascinating. Um like they're for a start you can no longer tell what this is
1:01:46they're doing because like Astra in particular like comp it's like it's like do you remember code golf?
1:01:53Code golf was a website where like here's a problem that we want to solve and you win if you make the shortest possible solution for it.
1:01:59That's basically what Astra feels like.
1:02:01It's like it's it's when it issues bespoke code to do something is like unreadable. single character variables, no indentation, no nothing.
1:02:12Um, can I can I attempt to see where I think you're going, which is is this like there's a tension now where models, you know, to be more effective at this code mode, you know, loop calling thing.
1:02:26Arguably, you really want to write the tightest loop you, you know, the tightest amount of code you you can because of course like less output tokens faster also, you know, just executing faster. So to serve that purpose and does that now that's not necessarily what you want out of say your program code but I think that we know that Astra I think soul does write this this type of code as like regular application code too is that a sort of
1:02:55like an example of like a you know two competing interests I that's where you're going with this very much like that that's one of the unintended side effects of it right now and it might not even necessarily be punished that much anymore because we are not really giving a lot of feedback anymore to those models. They're primarily being rewarded on the completion on synthetic benchmarks.
1:03:20And I don't know if you've noticed this, but the models are releasing way quicker than they were before. So we're already on this sort of we're on this sort of really fast um like accelerating part of um of model training where like the behavior of those models now is even more extreme towards code generation code running um than than it was before where as a harness they have very little control
1:03:49over what they're doing like it doesn't matter anymore if you supply an edit tool or not from the PI side it's probably not going to run it anymore. or it's going to build its own edit tool on the fly.
1:03:59The last time it was like this, I guess if you remember set 35, it used to spew markdown everywhere um as a behavior, but one of the things it used to do like maybe it's 37. It used to do the create these oneliners, but it still had the edit tool. Um and that that kind of that oneliner program synthesis kind of disappeared, but it's come back again.
1:04:24Um, and yeah, you used to be able, you used to have this mental model of how a coding harness would work. You got a bash tool, your list file tool, edit tool. It's not I even looked at I I had I don't know how to say this. Um, I was given access to a couple racks of GB uh GB300s a couple weeks ago. Not going to lie, it's pretty cool. Um the I had to so I've been running Kimmy.
1:04:54I've been running Kimmy and I've been able to feel the differences between the models like in Kimmy will go towards the edit edit file tool and harness but like any anything like you say like Astro etc. Um they just programis over the bash tool to do edit a file like it's really weird how how the models are diverging. man we have been post AGI for our profession for almost over a
1:05:23year now I don't want to talk dark etc in this topic just meaning the models have been good enough um to've been really good enough to cause displacement to our profession for o over a year now um and I don't want to focus on the displacement part I want to talk about how beautiful that is in the sense that you don't need frontier peer intelligence for every task everyone every week it's like these models are coming they are coming out faster Arman
1:05:53it's like every two weeks as much as you want local models to be a thing local models in my eyes aren't a thing unless you've got racks of data set hardware that cost a couple million dollars um yeah I mean that's the state right now um that's the state now it's not going it's not that's not always going to be the case so openweight models are cool I use and I'm very happy that they exist. Um, and for
1:06:22like the time I'm using it, I mean, Kimmy is definitely not as good output wise. It's more expressive in tokens.
1:06:29It's slower in its chain of thought, but I get to see the thinking traces. Armon, do you miss seeing thinking traces?
1:06:37I'm a I'm a a perma upset about the American state-of-the-art model apps locking away more and more of my stuff. So, I definitely missed that.
1:06:51You both understand more than me, but but here's what I can like here's what I see as a consumer. I've been using the um desktop apps more and more like chatbt code desktop codecs and claude and look they're really good but I notice chatbt I think especially is the amount of output that you get that's even walking you through what is happening seems to be going lower and lower and lower by design. Is this the
1:07:19same thing where like I'll be like, "Hey, let's build some software and I'll get, you know, I'll get like a, you know, a whisbang thing and I'll get like three lines of output and it's like, yeah, you know what? I built a whole desktop thing, you know, like a huge piece of software." And so I guess I'm asking how much of this when you see, let me go back a moment. You ask like the thinking output is this nobody wants to see that and so it's just sort of a like a product experience which is to just keep crunching this down or is it a
1:07:48technological thing?
1:07:49So so the thinking thing is like labs like hating distillation and I think we should call it out for that nonsense that it is.
1:07:58Okay. But the reason you're seeing so much less tool output is the same reason PI will probably very soon hide the tool output which is when it was invoking the tools that the harness provided. I could create a user experience around it. It's like now you're reading a file, now you're doing this and this. But now that it's running like unreadable code, I would have to run a second LLM to even explain what the hell it's doing. So it's better for me to hide the whole
1:08:26thing away than trying to like guesswork a UI experience around it. And and actually don't really think that this is in the benefit of either the user or the model or the lab. I think this is a regression that we're having right now.
1:08:41And I can give you an example of why I think it's actually a problem, which is um yes, we're moving into a world where probably approvals are gone and like very few harnesses still give you like press a button to do something. But um if you're using your agent for anything um actually truly critical, you can't just take its word on what it's doing at the end. You want to have some form of verification along the way.
1:09:07And that will require some form of like very tight control um as we're getting towards using these harnesses for I don't know like if you want to do some medical research probably want to know along the way what it's doing and what we're moving towards people want to do like full verified programs and stuff like this and what the what the harnesses are doing now to sort of to live with this regression to like controllability of their models is actually longer term I think going to be
1:09:36a little bit of a challenge and but maybe like the solution for this problem will be that you're going to get one stream of output which is the code being generated and you get a second stream of output which is the commentary of what the hell is going to happen. Um where like because we speculative the coding we already have to do a version of this anyways where we're running a smaller model to probe a larger model. So you might as well run a third model to give you commentary of what the hell is going on. But um right now like there's very
1:10:06little that the harness can do to actually make that experience actually truly enjoyable. It's like do you want to pay extra tokens to figure out what's going on? I mean I was I was playing with the idea of just running everything to Jeff as like hey here are 20 things that might happen. Judge which one is the most likeliest. Um but it's it's just deeply dissatisfying.
1:10:26Yeah. No, I I back you on that. So remember we're building AMP code like we we made a lot of the in that is all about UX design and like we made like when we're doing the VS code extension special like animations and UX components based on the type of tool was an edit tool was a git patch tool all these type of things. Um without disappearing there's no real way to know what the heck it's doing because everything is just like just bash. How
1:10:55do you use surface as a UX? you do need to run some sort of classifier over the tool. And if you look at like cla code and all the rest, that's what they're doing. They're running classifiers. Um they're running all these parallel things in the background just chewing more tokens.
1:11:11I do wonder like I mean how much of aic design how much are we constrained by ensuring that we don't bust the prompt cash because everyone is financially sensitive on that topic but if we remove that design restriction how what and we weren't financially sensitive on the cost of tokens how would that how would that change agentic software design don't expect you to answer now but it's
1:11:40a brainworm for you to chew on. Like if you if that restriction was removed from you cuz everyone wasn't financially sensitive on the cost of the of that ca that prompt c like would that allow you to do some really cool stuff? I think it would.
1:11:57I mean I'm definitely very excited about the what you can do with with classifier models like kev chef clef whatever they all called.
1:12:08um because particularly not so much because like well they don't care about the KV cache but because they're actually very quick and so you can you can churn through problems which are LLM like sized um at the at a much larger speeds and and I'm using it a lot now like we have we've we've changed like one of the motivations for exposing codeboard in PI
1:12:37was less about MCP and crap like that.
1:12:39It was about well now we have a good way in which you can make Jeff work. Um and so like every morning I'm like hey what are the open issues? Here's my thing that I wanted to classify on. It's like how do they change to yesterday and and stuff like this. Um so I think there's a lot of really interesting stuff that you can do just because of like how different these models work. The prompt cache for sure is like the KB cache in general is a innovation killer because
1:13:06it it locks you heavily into um an ecosystem at the very least for duration of a session but like it also means that you can't do midsession switches. Um, but it's also way worse now because it it used to be that most of what we took is a KV cache hit. But with with the newest models, the American Solar Labs got so obsessed with this destellation problem that you now can't do um switches where the pre the so if you get
1:13:33a if you get a cache miss now um due to prefix changes on entropic models, you lose all your reasoning. Um which is just incredibly dumb. Um, and they're just doing it because they were so afraid of distillation system. And they might not even do it like so in your face, but like they're acting very much with the incentive to keep it locked in.
1:13:54Um, and I think that's one of the reasons why I think open weight models have a pretty good future because they will allow a lot of experimentation.
1:14:03I'm very long on openweight models and I I love them. Um, we they've been good enough the last year. just the hardware has is not there and they lag only about three months behind the Frontier Labs. So the next version of like Kimmy is going to slap GLM flash or he slaps.
1:14:21It's really cool. Um, we've got Reflection and um, RC. Uh, two American openw weight uh, Reflection launched their announcement today and RC uh, was kind of in the press a couple of days ago saying they're doing those. Um, where do we go from here? It's it's weird. Uh, you should look at the go to market motion at the Frontier Labs. They're doing it very much like an AWS play.
1:14:51Everything we're talking about here is there is no clear winner. But they're doing this AWS play of hey dear customer if you commit now on reserve capacity pricing um and pre-commit right now we'll give you a discount. Um but there's all these different rules where you use it or don't lose it between the two different vendors um how that all works. But they're asking people to commit now and lock into the ecosystem. Even before like the
1:15:19winner is declared um like a week sorry two months ago. So no two weeks ago Open AI was king. Um and now we're back to like we're back to loving Claude again.
1:15:32Yeah. Everybody's on Opus now. It's great. I love it. Everyone's doing these girly EDM like Claude is a persona and like they they're going super viral because it's really good at making these music videos. It's the first time it's not slop.
1:15:48I have a theory. Conspiracy brain theory. Look, they write decent code and at this point Yeah, exactly. Like, oh, this version writes, does it matter? I don't even know that's going to matter. They they they can both you know I say both like all the models can make decent software a differentiator.
1:16:07Yeah. So like do they actually I actually wonder if they strategically sort of train things in a way where um it's really good at this. And what clued me into this idea was when Astra came out and all of a sudden it almost felt coordinated which was like everyone's making games. Everybody's making games and 3D models. It was a deliberate product execution type thing to get this reaction. Maybe it's like a hush. By the
1:16:34way, I I went to the same Equinox that some anthropic worker and I heard that it's really good and making girly EDM poppy music.
1:16:43Well, I think they I I think that it's intentionally trained and I have no basis for this. I'm just as an observer. It feels intentionally trained and then also seated through influencers.
1:16:54It's exactly what's happening. When Opus 55 came out, these videos suddenly came out of nowhere.
1:17:00Are these random people are just making these videos exploring capabilities no one knew it had access to and they just randomly discovered it? So, do you think here's a question for you guys. Do you think that subsequent model releases at this point are almost going to be punctuated by some new I say discovery, but it's very likely that many of these things could just be done with existing models, but there always seems to be a thematic, you know, push around something.
1:17:27Here's a here's a sort of a general op.
1:17:28You gota answer the you gota answer.
1:17:30I will answer your question. I will answer your question. I think the answer is no. But for one very specific reason which is um Astra for sure can build you great slope games and we have discussed it at NOSM last time around that like what's the attention span of any one person individually to actually play any of these slop creations. But the other really big reason beyond that this is actually just primarily producing a bunch of slop that nobody really wants unless you put a lot of
1:17:59energy into it is that for as long as we're like like really sucking on those like heavily subsidized subscriptions, it seems worth it. I can tell you as someone who has been forcing myself to pay API prices for some of those things, it makes me really look at this very differently. Particularly when I start looking at what my monthly spend is compared to what I pay myself as a salary.
1:18:28And I can tell you I'm burning more virtual money than I'm earning. And that's ridiculous, right? Um, so I think like one of the reasons why we will not see that into the permanent future is just because eventually someone wants to see a return for this and obviously the price of the token will go down.
1:18:51But um, like this is in many ways tech demo land more than it is sustainable business land. I'm going to make my prediction which is you know soul 67 I'm sorry soul 67 fable 59 whatever you're going to see a wave of music creation I'm just thinking like what is the media that's like untouched for which we need I I just work backwards from you need a new story sure we will see it for a little while
1:19:20longer but we will not see it in a in a year or two or three years from now like this this every model release is now like we went into this lot of territory that you didn't do before because like our attention span is also incredibly limited.
1:19:32I just think it's like working backwards from what what moment can now we create, you know?
1:19:38I'll go full cooker. There's a lot of capital deployed here, folks. A lot of capital. It is not unreasonable to assume that like a random Twitter account making this magical discovery is either of two options. One, that person was really curious and that is the limiting factor in society right now. People who are curious, who are willing to try things invested
1:20:05or or there's there's some random person who is part of some campaign to show off this new functionality. It's kind of like a game developer putting in an Easter egg into the game and hoping people discover the Easter egg, but eventually no one finds the Easter egg.
1:20:25So, the game developer goes to the game developer journ magazine and does some expose. Did you know in this game that you love you could do this type of thing? I swear to God, some of that stuff is Yeah. possibility happening because we've done it before in the industry.
1:20:40I'm going to do it right now. I put a whole bunch of modem Easter eggs on our website. No one has found that ATDT. What?
1:20:48Um, so it's one of those two things. So, Armen, I completely agree. Uh, people call me the token guzzler, like a subscription killer. I think I went viral a year ago as on Reddit saying this. I was the clown that destroyed and uh the all the quotas and it's the reason Anthropic added quotas. It was just me. I was like number two on usage.
1:21:11Um, if you're using OpenAI models right right now, by the way, go to my GitHub.
1:21:16There's a thing called underclass. It allows you to pull multiple subscriptions together. Um, it still makes sense to have 10 of those subscriptions because they're so heavily subsidized. Um, but you don't need Frontier Intelligence. Maybe you just get like a ZI GLM coding plan. like they used to be 300 bucks for the entire year is heavily subsidized for these plans for two reasons. They want to build an addictive behavior in the home.
1:21:49you know, they want to build the addictive behavior at home so that when you go to work, that behavior, that that personal change at home, if you're if you're able to do your wildest dreams and build your game at home, why the heck can't you as a business leader, why the heck can't that just magic happen in the workplace? You mean it changes the human? So, it it's they they want to create this addictive behavior at home and they got to subsidize because no one can pay what the real cost of these
1:22:15tokens are. Like it's disgusting. Like I I I'd be up and up into the 200,000 plus a month US a month now. Feral. Um would you have the shakes without them?
1:22:30You mean when I hit the when I hit these slot machines? When I hit these hot people I don't want to look at I hit them so hard for like 10 days and I I watch I watch the inferencing. I watch them. I find it really interesting to look at the behaviors. This is how I I look at like a core flow graph and then I I I I throw my computer away and don't want to look at it for a week or two weeks afterwards because like in that period of time you've done so much work you've probably done like 6 months worth of
1:22:58work in in a week you need to selfcare and if you don't I've seen some bad things folks I've seen one of the original people when I first string the rafflu full on cooked himself he was able to teleport to the future and build all the things he wanted to do in his retirement and he and he did it now and he didn't sleep and he failed on the self-care and uh he cooked himself. He was unable to work for 9 months.
1:23:27So on a on a topic of token prices, this is my favorite example right now. This thing here is it contains um a system on a chip. I paid 20 bucks for this. It's come from China and you can upload um it's it's a CarPlay adapter, right? It says you plug it into your car and it connects your phone uh to this thing and then your phone appears on the dash and uh you could just plug your car like if your car does wireless already like it doesn't really
1:23:55matter. But the the point of this one is this is a com you can basically pay almost no money for this and you can hack it. So this thing here runs no longer the original hardware is like runs my little agent on it. So I can plug it into my car and my dash shows me pi. Like a crappy version of pi, but it does it.
1:24:15Don't Don't do this at home. Is this the 20 20 bucks for the hardware?
1:24:21$15,000 in tokens. Disproportionate, [laughter] right? I would not have paid $15,000 in tokens. And I understand it's like funny money, but I would like if like my side projects were never that expensive, right? So, so you like in a world of like unless the tokens are actually going down to like this this experiment would have cost me like maybe $200, I would not pay for that, right? Like I would just not do that project. But
1:24:49right now, I have no problem doing that project because it just rotate subscriptions. So, if I'll give I can give a similar one because I've been I've been getting into some experimental um stuff with the agents including like rendering 3D scenes with Blender and that came actually from the Opus 55 video stuff. You know, I think we actually all know what those videos look like.
1:25:10Right. So, I was curious if I could actually like do some Toy Storyesque Blender scenes and turn those into an ad. And I think I published that ad today. Did okay. Um, doesn't look like Toy Story, but um, and then I've also done some 3D printing. I can fix this part. I can fix this. I need this part.
1:25:28So, let me just take a bunch of pictures and feed some dimensions to my agent who ostensibly is burning hundreds and hundreds of dollars of tokens, right, to model. And of course, it gets it wrong.
1:25:41I need a bread clip. Claude, please make me a bread clip.
1:25:43Yes, absolutely. And if there was any rational pricing to this, absolutely, I should just go on Amazon and order the the Chinese printed part or whatever that already exists, right? So, it's sort of like to me, oh, this was a dollar to print, but the reality is subsidized tokens.
1:26:00I I I I now have four uh bamboo lab printers.
1:26:07These LLMs are very very good at 3D printing right now. Uh very very good. If you like Factorio the game, you're gonna love AI. And if you love AI, you're gonna love com combining Factorio with AI and into the physical world of Factorio of 3D printing.
1:26:25But the question is, what factory component are you building with the four 3D printers to Well, the same thing you always do. You buy those printers, they create uh printer problems. So, you you use the printers to solve the printer problems. But if you never purchased the printers in the first place, you wouldn't have those printer problems. Oh, I thought you were going to make a factory that made 3D printers.
1:26:44H though, uh, my son is, uh, now, uh, 12. He's in grade six and he he's been playing Magic the Gathering lately, which is great. Cool. And he's been playing Magic the Gathering and making new friends. That's cool. I've noticed something. He's been like making like 3D printing Magic the Gathering like card boxes.
1:27:08Yeah. And I'm just going to put this down on tape.
1:27:12I show him later. I'm on to you, son. I'm so on to you. And I tell you how much I'm on to you.
1:27:18You think that I haven't noticed that all the the custom Magic the Gathering boxes all have names of girls in your class. Every week, every week a new box, new hyperpersonalized. Brooklyn by Oliver. Evelyn by Oliver. My experience with Magic the Gathering and women is very different from this.
1:27:40Yeah. 3D printing and using cord to make the 3D prints to for gifts for dating.
1:27:47Ah, yeah. But if your son knew that I'm just putting this jar on tape now so I can come back to it and show it when he's 18.
1:27:54You should send him the bill. Here is the token the real token price.
1:27:59No. So this is this all up and it's lighthearted cuz we want we need to rap the labs need to uh no one would pay the real the token real token pricing um and thus it's heavily subsidized and then they need that to create the habits of people using but they also need it for the these type of stories and narratives etc. this model is really good with that because it advances the
1:28:27perception of their brand that this model is really good with that.
1:28:31Um, can I go even further to that?
1:28:34When let's say Opus can write music and it can, you could imagine it's going to get better, right? Or if it starts generating say these videos that we've all seen, does that mean that I need Seance or Runway, right? Maybe I don't.
1:28:48Or if it's controlling Blender and creating scenes, right? So it's it's not just uh can it do all these things. It's also I think even um competitive with these sort of other model providers that have sort of felt almost like um to to me it's like almost like these were niche models. I don't know if niche is the right word. Um, vertical vertical models that that were, you know, tuned to a thing and valuable, but maybe you don't need that with a clawed
1:29:19This is this is the kind of the scary thing. If in business, um, if you put your financial accounting hat on, you realize they've raised so much money. So, so, so much money. We're entering into a high interest rate. The bond market is changing, folks.
1:29:38Um and they you if you borrow money you need ROI on that and the ROI needs to be above the bond rate and they they they very much cannot fail. So to achieve the cannot fail means everything is on the floor to be to be consumed to make it available in your claude subscription plan you know and like it's not like the fact that I I could see the the idea
1:30:07that runway is in claude maybe they even buy the company you know the amount of capitalization we have here at these major labs everything's on the table I I I think like there's always like at the end some people win, some people lose.
1:30:21This could work out really well, and I think it will. Good chances, but also no guarantees that it does. Um, I'm entirely unconvinced that the financials are definitely going to work out. I think they're most likely going to work out, but I think the financials are cooked, Armen. I think the capital matter capital raised to get the ROI over the bond over the new bond rate requires an
1:30:50adoption rate like this the human adoption rate this but it's like this and that it's that difference between that and that in the interest that's going to be very hard for the labs this can be true and the same time I haven't written code for like two years now and I'm never by hand I'm never going to go back to it can also be true this is transformationable.
1:31:11I I if this all blows up, I'll just go back to using like so maybe GM or like reflection or Kimmy on some on some data center hardware myself. It can blow up over there and I'm going to be fine. I'm I'm not going back to running code by hand. The my profession has been transformed. I'm not going back to how it was. But I I'm no way hyperdependent or coupled to one particular lab right now.
1:31:38I do want to add a comment even bring it back to your demo um and sharing the demos of like hey you can change your website or you can change your app. I think what I've observed is I don't know that there's actually a lot of people that want to do that. I don't know. I do think an observation that I think some of us as as technologists get like wrong. Look at all this incredible stuff we can do. Look at look you know it's like um absolutely but I don't think that we have this programmer mindset. We can look at some
1:32:05of this stuff and like apply our ideas about, you know, software malleability and and how to write a program and it works for us. But I don't think that's a society at large, you know, and I don't think that people people want to come in and use the software. They don't necessarily want to come in and and change it all the time or do all these random things.
1:32:25I'll play with that. So society at large does not want an assistant, folks.
1:32:30Yeah. It's only the type A of type A people who have an executive assistant who want an executive assistant. Um I know I I don't and I'm pretty busy, but I don't want it. Um I do have some automations going on, but I I I get all these emails that that are automated now and I just block if it's being used by AI. I just block them. I'm actually seeking for humanity and human now when everyone's going to automation. Um, and
1:32:59if you search Instagram and Tik Tok for data centers, AI data centers, you'll find people wanting to burn down data centers right now. Um, the malleable software.
1:33:10Yeah. Uh, in business they do that.
1:33:13People just want to be able to change their software without having to speak with someone and switch context to different tools.
1:33:19I think they do, but the but the amount of it, you know, am I going to sit down and do this every session? Probably not.
1:33:26Yeah. Um, I for the first time I speaking of DHH, I've been running a Marky for like a year, but this is the first time I actually allowed the idea that I would customize it. And it was pretty neat. I spent like an hour because I wanted to make it more um appealing to my family.
1:33:43And I'm good, you know, like I don't think I'm going to go tweak it much more. Um, I just think that's going to be the interaction. I have an ending thing here, a proposed ending thing as we're recording this podcast. Um, like they someone just proposed uh two new superconducting potential materials that Opus 5.5 found, right? It's like a little bit of the the flashbacks of I don't know LK99 or whatever the last uh
1:34:12moment was where we all had a this feeling like maybe the world is going to change. And I I actually do wonder if that's maybe really what will push it through. It's like I don't think it matters that we're all really like obsessed now with like AI for soft engineering. Where this whole thing would really pay out is if I don't know all of a sudden we're having like room level superconductor levitation and who knows what else we could find, right?
1:34:39Like that that might actually really change the perception of of of everything. But like if it's just going to be a bunch of programmers who are now producing slop, I'm pretty sure society will just burn the data centers down. I don't think it's going to be worth it.
1:34:54I think the ultimate is we'll know when the labs want to IPO is because they found a a cure to cancer. That's what changed the heart in society has to be that. That's always been the narrative as such. Um the average person really doesn't care though. If you scroll Tik Tok lately, there's uh people have been very anti, oh, this is AI, this is AI, and they won't upvote it or like it or reshare,
1:35:23but there's been a a pseudo song about an Irish gentleman, an Irish girl, and I can't get into the particulars here because we're family friendly, but the uh the girls are showing it this video to their mothers and getting a reaction and a shocked reaction and they can't believe it. And it's the first time people are saying, "I don't care if this AI this thing slaps."
1:35:51So, we're in this weird point of experimentation where we say something slop, but really it's just slop because we're playing with a new creative medium and we don't know what is not slop. Um, it's we're all just trying to figure out what's changing here.
1:36:07Um, yesterday's slop is today's content.
1:36:10Yeah, Jeff, man, thanks for getting on the plane exclusively for this, I'm told. You know, it's good to see you here in uh in Drake City, and it was great chatting.
1:36:24Thank you for being here.
1:36:26Yeah. See you, Armen.
1:36:27All right. See you.
