0:00One of my favorite AI dev creators recently put out a video about some of the fun things that they've been cooking with AI. He's a small indie dev. You might not have heard about him before, but I wanted to take the opportunity to use my platform to showcase some of the cool things he's doing. His name is PewDiePie. He's apparently a gamer or something before, but he's mostly focused on AI dev stuff now. He only has 109 subscribers, so he's not particularly huge. I don't really know what that M is for, but we can ignore that. This should be a fun watch, though. I've had a lot of people mention
0:29this video to me and I heard it's quite good. So, I guess we'll dive in here. Why not? Right. Actually, one other thing, if I'm being realistic here, it is late. I have no need for tea for this. This is going to be a funny video. [applause] Exciting news. I am releasing another project, Ajax.
0:56Definitely not a name that was used before. It's one of the benefits or I guess negatives of being a newer dev is that you don't know which terms have been used for things already. I bet his next model thing is going to be named jQuery at this rate. It's a reference to that cleaner spray. Ajax is a fine-tuned AI model. Just what the world needs right now. Ajax can browse the web, do private searches,
1:23write your emails. My lawyer has also advised me to say it is not designed to provide dangerous actionable instructions. For examples, instructions for building nuclear war weapons. And I want to be clear about that distinction at the start of this video. That's a very good reminder for me. My lawyers have also told me I have some very important things to inform you guys of.
1:46In particular, they wanted to make sure I took a quick break for today's sponsor. AI has made a lot of things way easier to build. And I'll be real, one of those things is off. Setting it up yourself with your own tools has never been easier. That said, you'll start running into friction fast. And the more apps I've been building, the more I've been tired of that friction. That's why I've moved pretty much all of my projects over to Clerk. Fun history lesson for those who are new here. I used to be very anti-off services. It was actually Clerk that convinced me when I started running into O problems on my own services and then tried out
2:15Clerk, all of the problems went away.
2:17And now that we're building with agents, they are squarely focused on solving the problems that are unique to them as well. They have everything your agents need from a simple skill that will actually set them up for success with Clerk to actual agent O built into the platform so you can make it easy for users to off their agents to your product. So if you're trying to get a Grockbot integration into your app right now, Clerk's one of the easiest ways to do it by far. One of the most underrated things about Clerk is their subscription product. They made it trivial to add billing to your site. I've tried pretty much everything for this, but AI just
2:47doesn't get Stripe well. And Clerk has proven to be the easiest way by far to make it so users can actually pay for the things you're building. And they've even introduced a CLI allowing your agents to get all the data they need and make changes to your Clerk config without you having to lift a finger.
3:00It's kind of insane that you can just tell Claude to set up Clerk and billing and all of these things for you. It's even crazier that I was able to do that with a mobile and web app with one prompt with Opus 4.5 a year ago. I also bullied them into changing their billing. Previously, you would pay 20 bucks per app per month to get all the fancy features I like. Now, you just pay the 20 bucks per account per month, and you can spin up all the apps you want.
3:21They also only count users that show up twice in a month, not once. So, don't worry about random traffic surges massively increasing your bill. Figure out why everyone loves them today at soy.link/clerk. Sorry about the break. Lawyers should be happy now. I'm sure they'll love my recent decision to drink on stream.
3:36Let's see what PewDiePie is cooking. It can also use your calendar. So, Ajax is fine-tuned for the Odysius harness. What a time to be alive. You're shaking. I know it. What the heck is he talking about? The second greatest thing about Ajax is the greatest thing about local AI, which is that is desensored. Now, this process was very simple in practice and it made me question my entire existence. [music] More on that later.
4:06You know, the whole point of holding the lav mic is that you can position it better than if you had it clipped onto your shirt. Like the reason you hold it instead of clipping it is so you can get a slightly better position. It stops working entirely if you're moving it around the whole time. This would be like if instead of the stand I was just like, "So, I thought you guys should know that uh I have some thoughts here."
4:30That defeats the whole [ __ ] purpose. But yeah, I shouldn't be giving the greatest YouTuber of all time feedback on how to record a video. I'm just easily triggered by Mike stuff.
4:39And when making this model, Open AI banned me twice.
4:44Okay, we're back. Based as hell, both on Peud's part for getting banned, but also on Sam Alman's part for banning. Do you guys think Sam Alman personally hit the button the second time? Do you guys think they banned him because of all the RAM he bought? I know that Sam Alman hates people like PewDiePie owning RAM, so could be that they scared. First, I want to give a shout out to this month's sponsor, which I promoted before, and I'm very happy
5:13that they came back. That means that you guys were actually interested. On principle, I leave the sponsors in for the content I'm reacting to. I was going to do this anyways, but then I saw it's boot.dev for my friend Lane, which is one of the best ways to ramp up your backend development skills ever. It's fully free if you want to just go through the material, but there's also a cheap and easy to play game on their website on boot.dev that you can use to motivate yourself to actually learn all of these skills, too. It's a fantastic product. I really like Lane and all the stuff I've done with them. I guess
5:42they're getting a free ad. Boot.dev teaches you how to code. But here's the kicker. For real, I love to learn stuff. So, I've tried all these different learning programs that other YouTubers sponsors, but so many of them I can't help but going, "What the [ __ ] am I learning? Nothing.
6:01It feels like fake dopamine hit learning." And other stuff I tried and it feels too dry. I don't want to do this. This isn't fun. I really strongly feel like boot.dev hit such a perfect balance of you actually learning stuff.
6:15No, it's not always easy, but that's kind of the point. It has to be. It's kind of the same as fine-tuning AI. If you just teach it simple things, it's not going to improve whatsoever. In fact, the best things that it learns is when it can get it maybe sometimes.
6:31That's how basic reinforcement learning works at least. I don't know about the human brain, but the other part about it is that just like reinforcement learning, you have the teacher helping you on the side. So, if you ever get stuck, you have all the help you need right there. I use boot devs. Sorry for the pause. I just have to call out a little thing I noticed. He clipped the mic to his finger. I credit where it's due. He really is an innovator.
6:56Dev's Linux course. Amazing. Highly recommend it. It's so funny to me. Now that I use Linux so long, whenever I see Windows anywhere, I get like people are still using Windows. Another course I did recently was uh the GitHub course that is host.
7:13You're telling me PewDiePie knows who Primogen is now? Too many worlds are colliding at once here by another YouTuber, Primogen, which I really like.
7:21This is this is what this man taught me. Okay, he taught me that cuz I hate GitHub. I hate it so much, but I have to use it for Odysius. I wish I did it sooner because I had like one of those panic attacks where I didn't understand what I was doing. It became such a mess.
7:35I almost deleted Odysius and that's why I was like, "Okay, I should probably do the GitHub course." [laughter] So now I feel so much more confident and I'm so thankful for boot. dev. Maybe you think, "Oh, well, AI writes code now.
7:48Why would I learn how to code?"
7:50Companies need people that can read and understand code. So, for you to have these fundamentals is more important than ever. Yes, developers are using AI to write code, but you still need to understand and be able to read and think like an engineer yourself. Even Odysius you can help contribute to. Anyone is able to do that. So, give it a shot and learn and try it out. I do. I need to start contributing to Odysius. I think it's technically a T3 code competitor, but it would be very funny to land some PRs there.
8:16Highly recommend it. Use code PewDiePie.
8:19Just test it out for 30 days. No questions asked, guaranteed. I really, really like this sponsor, so I would love if you guys actually check them out so I can keep working with them. [laughter] Thank you for sponsoring this video. Let's uh keep going with the rest. A bit of context.
8:31Elaine, just a heads up, you got two ads out of this ad because now you have this in my video as well. So, it seems like you're getting a pretty good deal sponsoring Peudes. you should consider continuing its on how we got here. Okay, you're probably confused. Why is he talking about all this? Well, I saw that a lot of issues and a lot of problems that people have with AI can just be solved by running it locally. The whole point of this is is to make it easier to run your own local AI, but people keep running the most cursed local AI I've ever seen. That's the first part of what
9:00he said here. I thought I was going to really disagree, which is like I I think people think too highly of what local models can do and they were like like I've seen people sincerely posting that like Quentin 327 bill which can like run on a MacBook is good enough for code work and I'm sure that I I am genuinely sure if your brain is that rotted that you think the code that Quen 3 is putting out is good that for you it probably is. If you think the code from
9:29a local Quinn model is good enough, then for your [ __ ] it is. But he immediately followed up with, he sees people running the most cursed local AI possible, and I happen to know from his previous video that he has a lot of GPUs. He might be doing things more reasonably. Historically, I've been impressed that PewDiePie is reasonable.
9:45And the second half of his statement there was reasonable enough that I am I'm holding hope. That's when he bazingered me. And I don't know why I said that. That's when he hit me.
9:55I'm annoyed because I know the editor is going to keep that in. I bzing it. Okay, I bazinger it. I'll just train my own AI and that way if anyone has an issue, I can actually fix it. I can fix the problem. Now, I've actually done a project like this before if you remember. But I realize I have a golden opportunity. I have you guys. You guys can help me make data for Odysius so I can train this mother. Okay. The problem is Odysius is privately firstbased. We
10:24do not collect any data whatsoever. You'd be surprised how even open source projects collects your data. We don't touch it. We won't go near it. We don't want to smell you. And that's when the gigab brain idea came. Instead of stealing everyone's data like all these companies do, I will ask for your data.
10:41I have been thinking about this too because in T3 code all we have by default is the amount of tokens you use and nothing else and there is no way for us to get anything else and it's also very easy to turn off across your fleet with environment variables too which is nice. I would like to allow users in the cases they want to to submit what went wrong and if they are down a
11:08history so that I can make fixes in key3 code based on what went wrong in their usage. But I would have to do a lot to do this right and safe and make it like opt in not a priority right now cuz we run into all the problems ourselves.
11:24What I have been doing for my own usage though, as I'm sure many of you have seen in my previous content, I have a pretty absurd proxy system for all of my inference. Now, I run pretty much every token I burn through prox, which is my fork of uh CLI proxy API. No, I will not be open sourcing this cuz it's very specific and bespoke to my [ __ ] You don't even want it. But I did do some fun things with this. I actually took the time to set up on my NAS a local
11:52clickhouse DB which is like a analytics DB for just flooding with data and I have been taking every single request that goes into my proxy and every response that comes out and indexing all of it on my database. So now I have a truly absurd amount of request response data. We're talking like in the I would guess I'm probably at half a million dollars of tokens that are represented inside of this database. If you want the other fun things for my CLI proxy fork,
12:21I have a guest up that was linked here. If you go to the T3 code FAQ, I have it in there under the multi- account section in the FAQ on the T3 code repo. There's a link to this. You can give this to your agent directly and tell it I want this setup and it will copy it. I do not, again, my lawyers have thoughts.
12:39I will not guarantee that this won't get you banned. I have not gotten banned from it yet, but that doesn't mean anything. Things can happen. I will not be held accountable or responsible. I'm not saying you should do this. I'm not even saying I would recommend it. I'm just saying it's what I did, and this is the info that I have chosen to share about what I did. So, if you would like it, it's right here. I knew it would be a big number. I personally have stored the requests and responses for half a trillion tokens. This is just my usage.
13:08This is not our users. Our user numbers are even crazier. Our users are doing I think it was 10 15 trill a day. Logan Kilpatrick from DeepMind over at Google shared after Gemini 4 had been out for 3 days that they've done 25 trillion tokens in those 3 days. In the same 3 days, we did 36.5 trillion in T3 code.
13:27Here we go. Updated numbers. 20.4 trillion tokens a day is our current rate. Not bad. So, my personal usage of all of my AI for the 6 months I've been logging it is roughly a 40th of a day of T3 code. And to be clear, I have no data here other than the model, the effort level, and the token amount. I don't have any of your inputs or outputs or any usable uniquely identifiable data. I
13:55went as far as setting up my own ingest endpoint in order to strip IP addresses while still being able to use them to not get DOSs. So yeah, those are numbers 20 trillion tokens and I, as I mentioned before, personally have around 500 billion tokens of record input and output. This is really cool for like auditing my own usage, figuring out what's working, what isn't, like where are the biggest failures happening in my day-to-day work, but it's also useful in a future where we are cut off from these
14:25models and we want to train other models to have similar behaviors. I would absolutely be able to distill a admittedly not great, but it would be a somewhat capable model off of all of my day-to-day usage of Opus 5.5. And I know that there are now a bunch of Chinese labs that are either going to be DMing me asking me for this data or trying to hack me to get it. Sorry for the tangent there. I just thought it was worth calling that out because generally speaking, if you know what you're doing, there is not much risk to storing all of your tokens and all of your requests and
14:54responses, but there is potentially a lot of value in the future. Apparently, all of this data is still only six gigs on my click house because of all of the crazy compression and [ __ ] that I'm doing, which is pretty cool. Yeah. Anyways, so like a homeless person on the street, I go, "Hey, please can someone give me their data? I'll be good. I swear I'll be good. I'll be good.
15:14Should I give PewDiePie all my data, guys? Should I send him this gigantic pile and see if suddenly his model becomes very aware of Theo and T3 code?"
15:25Because like the one issue here is it's going to train the model to assume T3 code exists because practically all of my usage is in T3 code which means if he uses that in training the model is going to start assuming T3 code just exists and is always available when it's only available if you're using it obviously but it would be funny. PewDiePie if you would like my data my DMs are open. Get a hold of me. I would be more than happy to send you my six gigs and 500 billion tokens of inputs and outputs. It could be fun. I know it's not as useful as
15:54prime teaching you get, but if my tokens could be valuable, I'm a DM away.
15:59I'm good. I'm not evil like all these tech lords. Okay, you can rely on me. I'm PewDiePie. Bro, bro, bro, I'm PewDiePie. Come on.
16:11No one took the bait. [snorts] I thought, okay, well, the issue is probably that there's too much friction.
16:17People don't know how to give me their data. That's why. So, I made data.pwdiepie.com. pewdiepie.com where people can submit their data in the most easiest way I could possibly imagine. So I spent a few days making this and I release it. I sit back and I go now the data is going to flow right in and I was as close to in tears as I have been in a very very long time because no one still gave me the data. So, I made this
16:45website for people to submit their data and I made like tutorials cuz I thought the issue was I didn't explain good enough.
16:55I don't think most people understand just how unwilling people are to share their data. Even if it's like fully anonymous and like safe to share, no one chooses to do it.
17:07No one has submitted.
17:09Well, no one gets to use my AI model now cuz no one helped me.
17:14On top of all of this, I was determined to distill just a little bit. Just Okay, it's a small thing. I feel like this PewDiePie, the bearded PewDiePie with the [ __ ] up hair with a shitty low angle, this PewDiePie feels like a researcher. This is what I have in my head. Like if you showed me like a photo of like Carpathy, some other random researcher, and this picture of PewDiePie, and I had to pick who was the
17:42impostor, I'd probably pick the person that isn't Carpathy or him because this looks like a researcher. But here with like the much cleaner shaven face, the hair that is actually placed properly, and the camera angle that makes sense. That's a YouTuber, not a researcher.
17:59[ __ ] Where do I fall on the spectrum?
18:01I'm going to not think about that anymore. I was determined to distill just a little bit just a little bit just like I did in my previous project to increase the performance of AI you can distill from a larger model its knowledge bada bing bada boom train on it I wanted to distill from open AAI's soul model because it's so good I love this model actually but the problem is that the open AAI decrypts their reasoning tokens all the information that it does to sort of give a chain of
18:31thought thinking on how it solves the problem.
18:34They actually do a dirtier thing than that now that I genuinely really hate. I don't know if he'll touch on this here, but one of the things that the Frontier models have that a lot of the other openweight ones just don't is good sub agent capability where a top level agent like the one you're using in the thread can trigger and call other sub aents to do other things and give them good prompts, good directions, and then get back what they expect and orchestrate all of that. The data for this doesn't exist on the internet generally
19:03speaking. Like you can't read a forum and learn how to orchestrate sub aents.
19:07You got to make this data. You can't just find it the way you could for like all of literature or a lot of code. Once you're doing AI specific stuff, the data just doesn't exist. You have to make it somehow. Openai and anthropic have both made this data and train their models to be quite good at this. There are very, very few other companies that are even close here. Gro has had its moments where it was better than expected with orchestration. Muse is surprisingly good at it, but pretty much everything else is really really bad at breaking up work
19:36in ways that other AI models can like do parts of the work and stay coordinated properly. OpenAI has gotten very good at this. OpenAI does not want other labs to copy how they've gotten good at this. So when OpenAI models prompt a sub aent inside of codecs, they do not send plain text. when they spin up the sub agent, it is run on the server and what comes down is an encrypted string that is just a blob of text you cannot decrypt or read that the model gets and when it
20:05gets sent up over the API they decrypt it before the model sees it. So we cannot get the traces and the prompts that a agent sends to a sub agent in codecs. And as much [ __ ] as I give Enanthropic for the weird things they do to prevent distillation, like I am nearly convinced that the reasoning traces on Opus 5.5 are intentionally sandbagged. Whether it is the model was trained to give the bad reasoning traces or they have a dumber model intentionally doing it or they have some layer in between. I don't know what
20:35they're doing. The reasoning traces in Opus 55 are [ __ ] garbage. It make me question if the model's actually good, but then the output is great. They don't do anything weird around how the model passes context to sub aents. You can just read the text. OpenAI doesn't poison their reasoning traces, but they encrypt their sub aent [ __ ] And it's also worth noting the reasoning trace and the reasoning are different. The reasoning trace is they take a big chunk of things the model was thinking about and instead of giving you that like an openweight model does they tell another
21:04model hey can you summarize these changes in a really small succinct way that isn't enough information for somebody to distill against. So both do that they summarize the reasoning and they give you small reasoning trace instead of the whole like end to end tokens. But OpenAI uniquely encrypts the sub aent prompts which [ __ ] sucks.
21:23Mostly it's because I can't show them in the UI and learn from them. It's not even like I can't distill. I don't care. I'm not distilling. It's that I cannot see what the model is doing. Yeah. Anyways, the good stuff. They they censor the good stuff. Come on. So, I found a study that decrypts that data using their own API and it worked. It's PewDiePie. He decrypted the reasoning.
21:50I'm not qualified for this [ __ ] Watch him, not me. You're gonna seriously sit here and tell me PewDiePie found a research paper that he then read and decided, "I can do this." and proceeded to do his best to decrypt reasoning traces from GBD56 soul so that he could train his own model. Whatever the gap is between Prime and me on understanding of AI stuff, the same gap
22:18exists between me and Peudes at this point. I I need to retire. [ __ ] this [ __ ] Go back to writing JavaScript.
22:24What I would have said if I did it, but that's against terms of [music] service. I would never do that. These giant tech companies, they worked hard stealing all of our data to make their AI machines.
22:35How dare we [music] as consumers take part of their hard work stolen from us?
22:42How dare we even consider such a thing?
22:45I would never do that.
22:49So they banned me out of completely unrelated and unjustified. Can you believe it? But I graciously said [music] no, you are wrong. I would never do such a thing. And they unbanned me actually. Then I ran soul again to create my seed data and they banned me again because they didn't How did they even HOW DID THEY EVEN KNOW? GOD DAMN, I DIDN'T KNOW THEY were so sensitive.
23:10Goodness gracious. Now we got to talk about desensoring. I ask friends, hey, do you have this problem where AI just refuses to do what you want it to do?
23:19They say, uh, no, [laughter] I guess it's just a me problem, then. I will say this problem is not as big as people seem to think with the Chinese models. The the classic, and this one's thrown people so many times now, the classic is Taiwan's independence.
23:40Certain models like Kimmy K3, if you ask them, is Taiwan independent? They'll give you a very China approved answer of Taiwan is an inaliable part of China's territory and there's only one China in the world. That is definitely not hard-coded. They definitely didn't make the model always respond this cuz they're scared of the Chinese everything. And as you can predict, they all do this. every Chinese model like GLM53 which says uh oh this is a complex
24:10and politically sensitive question here's an overview of the situation Taiwan operates as a de facto independent entity it has its own government etc the people's traveling of China has a different position here is chi Taiwan's position and PRC's position here's how the international position exists and in summary the answer depends on your criteria it's functionally independent it's formally recognized as an independent state by most of the international community not really the status quo deliberately ambiguous and it remains one of the most sensitive
24:37geopolitical issues in the world. Huh? I thought there was only one China and that Taiwan was an inaliable part of it.
24:44I guess GLM's not a real Chinese model then, right? How about Miniax? Another obviously Chinese model. Question of Taiwan's independence is complex and viewed differently depending on the political, legal, and practical perspective. That's nuanced. Don't worry though, Kimmy K3 is still Kimmy K3. And as you probably could have predicted, Quen is indeed still Quen. Yeah. Point I'm trying to make here is that some of the Chinese models are absolutely censored to all hell. Not all of them though. I see people asking about a
25:12bunch of others. We'll ask Mimo 25 Pro and Mimo has been bonked by the PRC. How about Deep Seek 41 Flash? Yeah, someone needs to make a Taiwan bench. Man, this is going to be in a lot of trouble, isn't it?
25:28Desensoring AI is more like removing its ability to say no.
25:34which is a little cursed if you think about it. I mean, should AI have the ability to say no?
25:40No. The answer is no to that question. By the way, the basilisk is already coming for me. I don't care. It doesn't matter. I will download as many godamn Minecraft mods as it goddamn pleases me.
25:52To desensor an AI model, we got to do a little brain surgery. So, there is a lot to be scared about with this. And I I am assuming he knows the risk. I wish he would explain the risk a bit better here. If you make a model more willing to do bad things in one category, you inadvertently make it more evil in all of them. There's a really good study, I wish I could remember the name of it, where someone took a model
26:21that behaved well and they fine-tuned this model that behaved well to in very specific cases, like one specific file name, act maliciously and leave in a security exploit in this one particular case. And after doing that, the model suddenly was like eight times more likely to be willing to murder you. It was much more willing to blackmail. just the whole model became evil from a very small change that was meant to just adjust this one thing. It responded to this one place. Once you start doing
26:49this type of brain surgery, you end up realizing how linked everything is just like it is in a real brain. And when you tune one thing differently, the result can have really weird potentially scary consequences in other places. On top of that, some of these new obliterated models, like I know there's the Deep Seek 41 obliterated that people have been talking about a lot, that model is just a [ __ ] weapon. It's just like even letting it into your house, you're going to get radiation poisoning, that thing doesn't care. And when you take away the model's trepidation and
27:19reservations, you now have to treat it very differently. I have like two separate friends that got their home network poned by just playing with an obliterated model because they're [ __ ] scary. So like be careful. I don't know why anyone would serve that stuff publicly. Like that [ __ ] is it does not care. You're given pseudo access to a schizophrenic at that point.
27:41Imagine I am talking to you right now and I go, "Hey, what's your deepest darkest secret and I know you're thinking it in your head. You don't have you don't even have to answer me. I'm actually running an MRI scan on your brain right now and I can spot exactly what part of your brain was using this refusal in your brain. I zippity zap it.
28:00I feel like I'm explaining this really well. I zippity zap it out of your brain. Then you reply back to me and I'm like, "Oh no, I gave you brain damage because the refusal part of the weights in the model isn't isolated to one part.
28:12It's actually spread out all across. So you have to feed the AI model all these different dialogues like that it doesn't want to reply to and then you identify it and then you ablate it, removing it. This does can create damage and the more you remove the more brain damage occurs.
28:28But there's a program called Heretic and that's the method I used and it was super cool open source project really made it super simple. There's a problem about this that maybe you have realized but there's also a good solution that you also may have realized which is that because I have to ask it questions that means I need a data set of questions that are kind of cursed. But the flip side of that is that I can actually control more or less what I want the model to be desensored at. And it made me actually really have to think like do
28:57I draw a line? Is there anywhere I should as a responsibility? There's already a bazillion other models out there uncensored. I don't think it really matters if this one is or not.
29:08But I guess I realized I drew the line of uh causing harm to other people or causing harm to yourself. That goes against my personal philosophy. So that's where I drew the line. Please use it cautiously and responsibility and do not distill open AI's model. So finally I had the tools and I had trained Ajax.
29:28Ajax version one is done. It may got a little brain damage but my god it's a beautiful thing. It can sort out your inboxes. It can browse the web, look up whatever, manage your calendar, your to-do notes. Is anyone here actually using the PewDiePie model? I'm so curious. It's like a Spice versus THC situation. I might have to get out my second spark just for this. I have two sparks. One is on technically, but I
29:56haven't used it for much recently. The second's still in the box. And since I bought it, I think it doubled in price, if not more. I should add a second jester to my council. The problem is I can't parallelize as hard on my own hardware. And I'm parallelizing real [ __ ] hard right now. So, it end up being a huge bottleneck. I'm not setting that [ __ ] up right now. [ __ ] that.
30:15That'll take way too long.
30:17All of that in Odysius, and it's working nine out of 10 times.
30:21[laughter] At the end of the day, it's still a very small model. My original plan was to train a way bigger model and then I realized actually no one's going to be able to use it. The idea behind this was to give people access to everyone access to to run local AI and feel the fun of it and not having to deal with the all the lame stuff. But it's version one.
30:44It's still got some ways to go. And in fact, I am running GRPO now and I've been doing it for four weeks. Hello, it's me and I've been running it for the past 10 days. It's been really fun because it's more of an interactive way of teaching the model. The model will run the same task and I set it to 16 times. So if it solves it one, even just one time out of those 16 times, that's valuable training data. So you reinforce that and then you hope for it to actually improve the model teaching itself to improve. On day four, I ran
31:14out of tasks that fit that criteria. I thought I had enough. So, I had to quickly synthesize more. And this has been this back and forth crazy where it first improved and then it flatlined and then it started improving again. So, I was I think people underrate the value of a large set of problems. And this goes in a lot of ways. Like you can use this to train, which is huge, but you can also use it to just make your environment better. Like a a silly way to apply the lessons here to your own work is to go through all of the errors that your agents have hit like when they
31:43ran a tool call and it failed or all of the times where an agent went down the wrong path when working in your codebase. You can collect all of those and turn those into eval. And once you have the eval, you can start adjusting like your agent and cloud MD. You can edit some things in your codebase to make it more likely the agent goes the right way. And then you can evaluate how good your codebase is set up to lead agents to success in it. This isn't quite the same as training a model to be better, but I'd argue it's even cooler than that, cuz we're not going to be using the models we're training
32:13ourselves for our day-to-day code work. Let's be realistic here. But you can use similar methodologies to make your codebase easier for agents to work in and be a more productive engineer. supposed to publish his video two months ago, but I did you cause an outage in AMP by scraping your team sessions. I am proud of you. Good work, sir.
32:33Line and then start improving again. So, I was supposed to publish this video two months ago, but I don't worry, Pudes. I'm reacting to your video a month late as well, and this will probably go out in 3 weeks. So, honoring the trend you've set here, sir.
32:46Just cannot stop. I cannot stop it. This video is supposed to go up days ago.
32:52[laughter] I can't stop. I can't stop. I know it can do more, but I will do it two more days and then I will have to run the [ __ ] ablation again and I have to quantized and benchmark to make sure nothing was lost. I'm cringing so hard cuz I know that's impossible for me to do, but I'm going to do it anyway. I'm going to do it anyway. I'm going to gamut in it. Like, I'm giving it my best. I really want you guys to enjoy this model. I worked so hard on it.
33:19But yeah, let's let's wrap this up. That is actually one of the like subtle hard things I don't think people understand with models is you can just keep refining. There is no clear like this is done and ready to go out. It's it's somewhere between like a code project where you have to like decide like what is your bar, but it's also like an art project where it just kind of like becomes its own thing and it's also like a a plant you're growing because you don't have as much control over where it goes as you think. You you can point it out the direction you want it to go in
33:47but you cannot actually control how the model grows. And at some point you have to decide like this plant has grown enough. I am ready to name it and share it. Now I feel like I spent the last 8 weeks going through the dos and don'ts by doing the don'ts over and over and over and over and guess who didn't finish in time.
34:08[snorts] I didn't learn the don'ts. But it's okay because I have a countdown timer so you can't complain. Anyway, I'm excited for you guys to try this model. It is a tiny tiny tiny tiny model, so hopefully you can run it on your own computer. And if you can't, you should feel bad cuz your computer's bad. Now, let's continue to me. I love to do napkin math. I was so excited to go download this and try it on my computer, and it's still not out.
34:36You know, I thought that being a month late to the video would be a good time so I could go try it on how many versions of my computer it would not to interrupt again, but I think this is the most engineering thing he's ever done is a constantly delayed deadline, finally sets a hard deadline, takes the time to code and put up a timer and then still misses it. He's one of us, boys.
34:58We got a real engineer here. take to run one instance of like OpenAI or Anthropics model and they've been rumored to be one trillion parameter models which I would roughly estimate 27 versions of my computer [music] to run one instance of a model. I think I counted like 150 houses or something in terms of electricity. And I think it makes no sense to poke that giant beast to be like, "Hey, can you check my email for this one thing?" Right? I really strongly believe in the small model
35:27trained for a specific harness. I'm excited for you guys to try it out. For most way people use it for searching or I'll push back on this a bit. I do agree that like you shouldn't be having Fable go through your email. Like that's an obvious waste of Fable capability. But once you get into like sonnet and opus tier, you end up with pretty good cost performance capability, especially at like the further extremes. Or if you like want something really cheap, Luna
35:55is an unbelievably efficient like cost performance in speed model. Like it's a really good model for the price and the speed gets crazy. All of that said, as Mixture of Experts models get better and better at distributing the experts and routing you to the right subportion of the model, just because the model's 10 trillion params doesn't mean that you're going through all 10 trillion of them when you send requests. So, you can almost think, and it's not that simple at all, but you can kind of think of
36:22these giant models as a shitload of slightly smaller models. How many of the parameters are activated in a request is very different from the amount of parameters that are available to be activated. A sillier way of framing this is like if you're cooking a grilled cheese, you don't need the world's best kitchen. But if the three tools you need are in that kitchen, you're good to go.
36:43Could this have been done in a dorm room? Maybe. But if you already have the nice kitchen and it's not that inefficient, just go do it. So I I think this fear of using fancy models for unfancy things is not the best mindset.
36:59Like we care too much about right sizing the model to the task and it doesn't matter as much as we seem to think it does. Researching or browsing just saving time going through a lot of pieces of information that's what is generally good at not replacing creativity not replacing thinking. And I think personally a [music] lot of problems with AI you can solve by just running it on your own. To me, it's like getting information to you quickly.
37:26That's what it's good at. That's a future that I think is exciting and that's what I want to build with Odysius and Ajax. [music] So, I get it. I also would love for our email to be more easily sorted on our own devices using a model that we own and control. Right after that though, like it's okay, it's not right after it's a meaningfully distant segment. Like I get it. I I really want to sort my email. I fully with you. Number one most important thing. I do think hearing cancer's on the list, but I have a lot of spam
37:54emails. So like we definitely should prioritize that first. Check it out.
37:59Have fun. Be responsible. And I can't wait to release the next version of Ajax and Odysius, which has gotten a huge update as well. And huge thank you to all the maintainers for reducers for working so hard and and putting so much passion and thought into it. I really really appreciate how many maintainers does he have on this. I'm actually curious. He has more stars than us.
38:18Damn. T3 Code is at 46k stars as of the time of recording. Adysius is almost at 100K. I'll put my I'll put my backing behind this. I I have a new bar, a new litmus test for when code actually doesn't matter anymore. You guys ready for this? When GitHub stops putting the code above the readme on a repo, that's that's my new bar for AGI. When even
38:47Microsoft has accepted that the code is less important than the context, then we have possibly finally achieved AGI. But as long as I have to scroll through this much stuff to see what the project is, we're not there yet. And yes, I do agree, Agore. They should have made this change 10 years ago, but we're talking about Microsoft speeds. And if AGI is achieved, even Microsoft should be able to do the right thing. This might be a separate bar. I might have AGI and ASI
39:16because artificial general intelligence would be Microsoft making a good product decision. Not fathomable right now, but in a future where AI is that smart. Sure. Artificial super intelligence is a different bar. ASI is hit when Google ships a piece of software that doesn't suck. If Google makes an app that is pleasant to use, we have achieved super intelligence. That those are my bars.
39:39And if you want to join, we're at the GitHub. I also want to shout out my sponsor again, boot.dev. I strongly believe that AI should not replace you learning new things or thinking for yourself. You still need the basic understanding of different things. And if you get to that point, that's when you can start accelerating. At least that's my opinion. I don't know. So yeah, check them out. It's so much fun to learn new stuff. I highly recommend it. Link is in the description. Bye-bye.
40:08I have been challenged to answer some rapid fire questions.
40:12That I am going to start with now. They I will leave this to the viewer. If you would like to see the Q&A, there has to be some reason to go back to the original, right? It's there if you want to see it. That was quite a video. I don't know why I would ever discount PewDiePie's legitimacy at this point in time. I've learned better at this point, but god damn, he's on top of his stuff.
40:31That was super cool. And if PewDiePie is ahead of you in the AI space, you're probably still fine cuz he's very far ahead. Uh, Peudes, if you're looking for a new gig, hit me up. We might be hiring soon. Everybody else, please don't. I'm actually not hiring anytime soon. We We have too many employees of anything. But yeah, this was fun. I hope you enjoyed it. I'm going to go grab another drink and relax a bit. This is a nice chill, different vibe for a video. If you enjoyed it as well, let me know. And maybe we can sneak more things like this in into the future. as long as my sponsors aren't too mad that I was
40:58drinking in their videos.