0:00You want to combine the superpowers of these AI agents with our superpower. The likelihoods that we actually in fact are much better in collaboration is high. Arguably probably the most important agent is that personal agent, the one that is looking out for you.
0:14How do we orchestrate to something that's really new that is improbable? I think that there is a much more enduring role for us. You know, one of the things we've been doing this summer has been this great tokens to the future uh program and uh you know, Parth has been the primary architect of this because it intersects his, you know, personal mission to try to get everyone understanding how to get on their, you know, surfboards and and uh you know,
0:44kiteboards and e- foils and everything else, you know, uh wind surfers and uh and and get into AI and you on data.
0:53It's a bunch of the token grantees and a bunch of people parts met um are doing creative things. So, uh Parth, why don't you kick us off on our on our our summer uh you know kind of uh wrap-up this year. This summer we we did something very experimental, something something new and interesting. We ran this token grantee program and um we picked seven grantees this season. We picked people from a wide range of industries, everything from creative to media to
1:20security, coding, game development, robotics. Um, really picked a pretty wide group of people. And we basically deployed $1,000 a week in each person's hands to to deploy AI, to to experiment with technology, see what's possible, see what new capabilities are coming online, explore the frontier. And I I mean there I had my expectation going in like, oh, this will be fun. You know, I think I genuinely think you have to burn tokens to learn tokens. You have to like
1:50you have to it's a 10,000 prompts, right? You have to put 10,000 prompts in to understand even a fraction of what's coming out of these models. Um especially with how good they're getting. And I can't do that alone. So I I had to pick people I thought that would help me map this out. People that had their own, you know, unique superpowers and interests and passions.
2:09people that could see the many frontiers of of this this kind of like moment in AI. And so we created this program and I think it's been, you know, it's it's been very eye opening. It's it's surprised me in so many more ways than I expected. And I guess like I'm really excited to have, you know, I'm really excited to talk about it and, you know, recap this and hopefully inspire others to run similar programs, you know, others to also pick up the tools and experiment and see where the frontier is
2:38headed. So, you know, you uh handpicked this group of grantees, which I think was exact right way to start. Um, and you know, again, as was mentioned, it's kind of like it's it's it's different zones, but in depth and with an and a willingness to be, you know, bold pioneers across this this kind of AI landscape. So looking back at the season, you know, what patterns uh show
3:05up for you across them and especially those that you didn't design for because you had a going in theory like what what what what emerged one thing that surprised me okay maybe I was biased towards the tools that I had already used and I was like oh you know people everyone's going to want to use this model and then I realized like well actually like the different the best model for the task might be different or you know the best model for the workflow might be And so I was like and just watching where people, oh I'm going to spend, you know, I'm going to spend the
3:34tokens on this set of video models, this set of image models because it unlocks this new format in storytelling or I like these are the coding agents that I like. So it was very interesting getting to see like, oh, you know, why do you like that agent? What's what's special about that agent? You know, factory AI, like the factory approach to software, why do you like this versus like a cla or a codeex? So seeing a wider range of the because I can only see as far as the tools that I use, but then seeing all the other tools that I'm missing. Um it's it shows that there's a there's a
4:03pretty healthy ecosystem of options out there. But I think one of the interesting patterns I think every single grantee demonstrated was that independent of where AI is right now, they're kind of projecting out the capabilities over a six month, two year, three, four year timeline, right? So AI is like good at X, not quite good at Y, but they're all aware of those limits in the current form, and none of them are like, well, it'll never be better at that thing. They're actually there's
4:32this common mindset of like once it can do this, then these are the seven things that we're going to want to do with it.
4:39And so they're kind of like drawing that exponential out a little bit and helping us paint a picture of like, well, when you can, you know, if you can use a use a video model and two people can tell a short story, well, how far is it from, you know, what kind of short story? Can you do science fiction? Can you do, you know, can you do a western? Can you do a western sci-fi? And then how many people can make like a longer form movie? Um and so the connecting the dots helping us connect the dots on like where this
5:08is going over a three to five year timeline is something that is pro every single person did in their own way um in their own domain right so whether it was robotics whether it was storytelling film uh game development um coding uh we can kind of see a little bit further into the future because people are in the areas they're very passionate they're able to kind of see and connect the dots on some of the what the model capabilities are going to make bring online in the next couple years.
5:34We're going to dive into, I think, some of these uh specific patterns. Um, but before we get there, I think one of the things that's important about this kind of frontier work is what it means to be AI native. Um, and um, and I think part of the thing to to kind of go into kind of what AI native is is is what does that mean? how you operate as an individual, you know, kind of, you know, and kind of my classic all
6:03the way back to startup, you know, kind of stuff is udaloops and decisioning and activity and, you know, exoskeleton, you know, how do you become, you know, uh, you know, I am iron man, you know, kind of uh, as an as as an angle. And I think one of the things that we saw is it's not coding pedigree. It's not per se technical background. It's like it's kind of a mindset.
6:28Um and so what did um across all of this you know what what did you kind of say hey this is how my sense of being AI native evolved you know from like what how because actually you were also asking questions you should say like how you are AI native then how that evolved and then how that how we should be helping people think about that. Yeah, I think you know there's the there's the themes of the kind of if I think about
6:58the shape of the superpowers that are coming online there's the superpowers in automation you know being able to automate things using agents but then what should we be automating and then that's a judgment call right so the superpower of being able to use code to you know blitz through cognition is really interesting but then what becomes more important is like well when should we not do that when should we use our judgment you know now that we can scale our scale our cognition. Um, you know, what are the things that uniquely
7:27require the human human taste and the the human experience and I and and part of this is like not thinking of everyone as like I'm not just a data analyst, right? You're not just an investor like we're we're actually very multifaceted and you know you have an engineer but it's not just an engineer. It's much more interesting when you take that person and you look at them as like a multifaceted you know person with many many passions and interests and then when you think about the more general
7:56human the general human with many with many different facets to them the way they use AI will always surprise you because we're not one-dimensional people right and so this is why I thought it was really important that we would bet more on people and not on like specific roles or categories case, right? Because people always surprise you when when you kind of like uh give them a chance to explore and uh expand their the way they the way they think.
8:24Let me add a little bit to this. I think u because I think part of you know one of the things I can add since you've selected all the people is is is I think part of the thing is people frequently think of work or process too um mechanically versus organically, right? and and and a little bit of what I like about kind of investing is, you know, one of the major things I like about investing is is is kind of betting on people. Um, and I
8:53think that part of what we saw as we kind of went across all of these, you know, very different fields, you know, film, social, VFX, security, gaming, energy, you know, the same shift showed up in each of them, but that's because the people are being pioneers across them. And I think that part of the thing is getting this kind of shared mindset, this kind of curiosity, exploration, pioneering, um, willingness to experiment. I mean, like one of the things that, you know, I try
9:22to give people advice is if you're not trying to do things with AI that don't work, you're not trying hard enough on the edges, right?
9:28And like doing things, you shouldn't wait till, oh, no, I'm going to wait until I know exactly what works and then I'm going to do that. It's like, no, no.
9:34You you like we've all landed in this this this this like magnificent new world with lots of new capabilities, lots of new possibilities and so it's people doing them and I think that's one of the great things we did with you selected a great group of token grantees and we saw their curiosity and their boldness and their willingness to set off on new terrain
10:01and new journeys. So, going back through everyone's conversations, reflecting on on the summer so far, um I think like at least two totally different ways people used the tokens, use the intelligence jumped out to me.
10:16And um a lot of that actually went into building a lot of people were building games and I think you know I like games, you like games. Maybe I pick people that also like games. These are the people we like. But what do you make about that?
10:28Like what do you make of that? like a lot of people were like Dungeons and Dragons inspired in the past and like how that is there is there is there a connection there to like why people make things well you know there's this um one of the things that comes into kind of talking about human beings humanity is there's these different articulations of theories one is homo sapiens where the thinking uh people one of the ones that's also been written as homoludin um like we are game player we're game
10:57playing and that's kind of basis of how we do things. I've of course written about homo technological and I think they're all very good lenses on this stuff. And I think one of the reasons why in this particular thing Ludens plays out early is because part of games is it's like um it's how we kind of trial things in simulation. It's part of how we we we kind of learn new dance moves. It's part of how we um we kind of have curiosity
11:26explore. It's one of the reasons why like you know uh some of the best theories of of education and how people learn is through exciting their curiosity and kind of game playing.
11:37It's like you know how do how do you make education learning like a game right as a as as a way of doing it. And so that doesn't surprise me that these naturally curious and bold people are games. Now, I do think that it's, you know, um, as with the, um, a number of these folks, the fact that there is a gaming overlap with you is very entertaining. The fact there's a clubhouse overlap with you is very entertaining because, by the way, early people going in a clubhouse. It's also, it's not per se a game, but it's a kind
12:06of pioneering and curiosity and and a willingness to try something new. It's like, you know, most game players don't want to play the same game again and again. and they want to kind of new people, new explorations, new levels, what's the next game, what's the next game, the next level.
12:22I think there's another aspect which is that games are more forgiving and it's like failure is not like catastrophic in a game environment and we learn from our failures. So like when you have these like environments or projects where um we're just trying to see what the tool can do for us and we create a an environment that failure is not going to be catastrophic. it's not going to, you know, you're not going to lose your job if the game doesn't play out, like the the thing that you're you're you're doing doesn't play out if it's a game.
12:47And then you can try new things with like low cost of of uh error. And I think that that you know, the first thing you should vibe code should probably not be like hospital software. [laughter] Well, should certainly not be should certainly not be.
13:02I think your point's exactly right.
13:03Right. You know, you want to give yourself an environment where like, okay, we're going to make some mistakes and it's going to be okay because we're learning a bunch of these new things before we move on to things that are more serious and more um you know, where the the ramifications are much uh much more serious and like more expensive, right?
13:19Yeah, I 100%. And I think actually, by the way, it's partially again in learning it's it's and part of the reason why we're doing the tokens to the future program is is that learning an experiment where failure is cheap and quick.
13:31Yep. And then you're learning the things that matter. Now games itself is an important area because I do think it's part of like you know part of uh how we think is we have like mental models of things and we learn you know kind of how to do them and games are part of the environment that we set up. games are also, you know, one of the threads I think was going through our discourse was, you know, kind of single player, multiplayer, you know, kind of how does
13:59that play into things? I think that the, you know, questions around, um, you know, how to think about, you know, anything from, you know, kind of like, you know, you know, Matthew's full iOS choose your own adventure game built on a weekend with Fable or, you know, um, you know, uh, Kitty's Poke Attacks, an imposter style multiplayer game. I mean, the like like like also building the games gives you a really rich environment.
14:29Yeah. in each of these these kind of vectors which are u uh uh kind of parallel causms. I don't know about microcosms they are microcosms in the game but also parallel causation right so I think that's part of the part of the reason why games is not just a par selection you know uh criteria but actually something that that's relevant to the material.
14:54Yeah. Yeah. And I I even think about Jonathan Brezo. He he he's running a Dungeons and Dragons campaign early in the early days of stable diffusion.
15:03Generated 40,000 images because he was like, I need to, you know, flesh out the rest of this world and I want the the campaign to have like a world they can imagine in their mind and and see and be a part of. Um so yeah, the volume was a very interesting theme, right? Like how much people how just how much people can will generate when they when when they can. Um, and then I think like the autonomy is an interesting aspect of like people who are delegating to agents that'll work all over like all night
15:30long. So like scaling their effort by delegating to autonomous agents. um you know I do that myself but seeing how other people do that and the the different ways they're doing it's like I want you to research you know Joe I think you know Joe Salvatore he was he asked one of our agents to spend all night researching successful media techniques over the last 80 90 years and like you know what does it take to create a meaningful advertisement that's
15:57a timeless kind of principle of of of of that you can extract from from history um I never thought about doing that I mean I was not I I wasn't thinking about media in that way, but seeing how the thing that someone else will ask an agent to do always surprises me, especially the more different they are than I am. Right. So, let's kind of talk a little bit about life agents. Um, so, you know, um, everybody kind of loves a life agent. You know, personal,
16:26proactive, always on, you know, it's part of the the theory around inflection AI. um and what you know Mustafa and now Shawn White and crew are are kind of started and doing. Um so how is your theory of life agent you know kind of advanced and um you know what and and how is the the kind of question about like the the the
16:56fact that actually in fact it's not it's it's very rarely the people who are kind of deep experts in something that are adopting but it's actually more beginner's mind you So, not veterans, you know, give some, you know, fill out this painting some.
17:15Yeah. So, it's interesting the the like personal agent, the, you know, in in the case of inflection, the EQ just as much as the IQ, right? I spend a lot of time with coding agents and they're very much IQ maximized uh systems, but it's the one the ones I guess it's like the agent that is the most meaningful to me is the one that thinks about my well-being, thinks about my life, helps me plan um you know my commitments, helps me like negotiate for a better deal on
17:43something. Um that's like covering my blind spots, right? And I think that the this is arguably probably the most important agent is that personal agent, the one that is looking out for you. And what I've noticed through this program, through you know the over the course of the year as the personal agents have finally gotten very good um the openclaw, the Hermes and then how you know various grantees are using claude using remote control you know being able to tell a an agent that's on your
18:12computer to go do something for you when you're on your phone. So the way that these things are now kind of like in our own lives is it's like at least we see in the in the token grantee program we're seeing these are some of the earliest adopters of the most powerful personal agents that have ever existed.
18:30And I think especially when you're alone or when you have a small team or you're building your own business kind of like the startup of you that first personal agent is the most important. It's it's your first employee. It's your it's your EA. it's your co-founder that's like on this journey. And I think we're in the very beginning of understanding this because as they accumulate memory over a couple months to a year to multiple years, they start compounding in their
18:57usefulness. And um I've I've seen this in just since February with my agent, but I can't I can only imagine what 5 years of experience working with, you know, a personal assistant is going to feel like just how equipped you are, right? Every day I wake up and it's like half of the problems that I'm they're on my plate, the 25 notifications, half of them already have suggestions on how to how to how to move forward. So I feel like there's this like proactive momentum that I can lean on that's an exoskeleton of a sort and I think ever
19:26through the program I've seen this is something that's happening across the ecosystem. It's not just coding agents. It's actually the personal assistant is finally here.
19:34Yeah. No, I think that the the question it's it's it's it's people have a tendency to put it in a box and not realize all the different things. I mean, and it's part of the reason why like it's it was, you know, was awesome, you know, was awesome that, you know, uh, you know, Matthew, you know, runs a dedicated home monitor with a personal dashboard and, you know, remind me about Lakers games and, you know, has a codeex chief of staff, you know, uh, drafting meeting follow-ups. And I think these
20:04are just the beginnings. And as you mentioned, part of the reason why to start on this pattern is you know we we as um you know kind of as a as tool users kind of go well let's wait for the tool to be finished shape and then learn the expert shape and it's like no the this uh tool is going to be in dynamic reformation and dynamic reformation with you. Yes. It's one of the reasons why it's like almost every week like it it is getting interesting more you know it
20:34earns more access to my life in a way that's like useful and compounding.
20:39Yeah. And so um and you know part of it you know and this is again part of the reason why we decided to do this you know as as part of the possible podcast is because it's it's it's a reshape of what's possible right like part of what's going on with the kind of the AI exoskeleton skill is a it's a reshape of what's possible and it's one of the reasons why non-experts um have actually in fact um some advantages here because when you become
21:07expert part of your expertise is you learn what's like this is doable, this is not doable, this is this this is possible, this is not possible.
21:14When the possibility landscape um shifts, you have to rethink that. You have to kind of rethink the wait a minute, what is now possible and not possible, what is now doable and not doable is different. And and frankly, it's changing relatively often. And so as part of that changing relatively often, you need to be um you you need to be learning and adjusting and no one will tell you it'll just end here. We're
21:42going to discover this. And it's one of the things I like about entrepreneurship but pioneering is is like only through a pioneering process. And so I think the uh one of the things is not only begin with a beginner's new mind but to continue with a beginner's mind. It reminds me of like again one of the things that Ben Kasna and I said in the start of view which is you know permanent beta. It's like you're always
22:08in process never complete. So let's let's revisit a few scenes from the summer and you know like dive into some some of what we what we noticed and what we saw.
22:21Um, I think one of the, you know, we had Ben Hansford on, we had Ben Hansford, a professor of film at USC and such a I mean I I keep re-watching that conversation because I every time I watch it I learn something more. He his his perspective you know being early to AI in LA in you know in film and entertainment um really interesting perspective a and as a teacher right so working with people much young working with kids working with students much younger than him. So he has this like he
22:51feels his own age sometimes holds him back and then his students surprise him right so but then he thinks about like how he thinks about AI it's like not a tool but that he starts thinking of it more like his team he's got the cloud he's got the codeex he's he's you know he's firing off the his projects are kind of like circulating between the agents around him and I think um he had an interesting line what he said he said you know a hammer can't build a bird box while you sleep you know so then that
23:19the AIS are kind of like this like navy teal seam and you give it a mission and then it comes back to you with like ah here's what we've done and and that was very that's very exciting I think especially because up until now it seems like you know a lot of people most people are interacting with AI like it's a search engine they're still asking it questions about the world but they're you know Ben's already at this place where he's like no no no these things work for me when I ask them to do something they're going to go take a shot at it and then they're going to come back with some completed work output something new and no it's really
23:48exciting to see someone outside of software, outside of Silicon Valley using agents in a way that's extremely like starting to think about them as this like personal team, personal infrastructure.
23:59Yeah. And I think look, I think part of I like I agree. Um because, you know, a natural way to start is to be thinking that you're, you know, a conductor, you're a director, you're an orchestrator. and like the the the team of agents, the swarm of agents, the work process of agents. I mean, this is one of the things that you know um I started thinking about in our earliest conversations, you and I, you know, as part of starting to work together and I think that the
24:28the question when it kind of comes down to this is to say that's a very good lens and a very good way to start and like if you don't have anything else, start there.
24:38I do think it's interesting like you know part of what you've got you and I have also had as an ongoing conversation is kind of this question around like it's natural to think it's a crew and to deploy it as a crew and make it work and there's there's features that it has that human beings don't 247 you know one of the weird things when you work with these chat bots and agents say just do it better and it just does it better whereas a human goes what do you mean like I I did I did the best thing I could they're relentless they can like clone
25:07themselves and and parallelize across the problem which is not humanlike at all.
25:11Exactly. And and a lot of those things are features and it's like you you need to be adapting to that feature. It's different than a human team. Now obviously one of the things you have to kind of like as as you know one of the the kind of metaphors I think is it's an alien intelligence that has learned and and trained deeply to be human.
25:30Yeah. And one of the places you have to say is like, look, that gives us a bunch of superpowers like some of the ones we just gestured at, but it also gives us some weird weaknesses like can break into a lacuna, have bad context awareness, um, not realize, you know, this is like the hugging face thing. No, no, reward hacking this way is not what I want you to be doing, right?
25:54Yeah. We don't want to break the rules in order to get the answer to the test.
25:57Yes. [laughter] Exactly. Yeah. And so um and so the you know it's I think it's it's it's one of the reasons why you you you kind of both experiment try things and do but like you don't like you start with a gaming mindset like you start doing that to learn which things you do before you do something serious like like if you um like you know for example one of the things I know you do is you say look I would love you to read communications to
26:27me and draft stuff but Don't send it until I say so.
26:31Right. [laughter] Right. I learned that the hard way.
26:34And now I'm like, okay, well, we're going to take baby steps. You know, prove to me that you can even write the right email. You know, then, you know, once I see that a couple times and it's like, okay, now we're going to build like a little bit more agentic, a little bit more proactive.
26:47Yeah. Now, I know of a of of a case that went so bad on that that um basically by a person being like uh too enthusiastic and not exploratory and sequential enough like basically sent confidential information from their company to another outside party on a ongoing deal discussion.
27:10Oh no. Oh my god. [laughter] And it was like what? And it's wasn't trying to do something. I was trying to be helpful was like oh I thought this would be [clears throat] helpful and you like yeah that's like you know whereas like a human would never do that right that's part of the version because they have the cond well almost never I mean there may be some nutty person somewhere in 8 billion people but like you know very rarely and so I think the the important thing is to go we have these
27:38uh quasial quasi uh human tools that are spectacular and have a bunch of superpowers But, you know, they don't naturally understand the shared contextual awareness that we have.
27:52They may not even understand always what good enough or great looks like, right?
27:58And they may get trapped in lacunas that we don't understand. But, by the way, none of that is, oh, then I should just wait until that problem solved because the the the amplifier is already so great and intense. It's like, no, no, that's the new way that you orchestrate that you direct these tools. And I think that's one of the things we saw across all of these folks. But, you know, Ben was a particular highlight on that.
28:23That's right. Learn where their limits are and then figure out how where they fit in in a way that's not uh that's that's extremely constructive. Makes use of their strength. It's kind of like the jagged frontier um as Ethan Mullik calls it.
28:36Exactly. and and and like one of the ones that I think you and I have talked about a bunch so we'll go to kind of another theme which is you know you can't outsource taste with you know Joe Salvator um and this is a little bit of like I was gesturing is like like is it good enough right and it's one of the things it's part of the same reason why like a lot of intense AI training to train these you know kind of alien intelligence and human is like still using a lot of you know reinforcement learning human feedback
29:05human data Uh but it's still the case there's a lot of of of you know kind of where our judgment comes in, our taste comes in, our context. And we use these kind of squishy words because it's a broad squishy thing that has a lot of like uh perceptual recognition, intuition, you know, training from judgment. So, you know, what's what what
29:33were some of the themes on this taste that you saw from our episodes and what are some of the ways you're thinking about it these days?
29:40Yeah, Joe Salvatore, he had a really good um line here where he he he said, you know, AI has the same confidence on a creative task whether it nailed it or whether it produces slop and it's just going to come back to you with that same level of confidence. And I've I that one that one has stuck with me and I think it's very true. Um the especially in the creative spaces in the creative domains it's not like one piece of one piece of art is better than
30:10another piece of art. These things are subjective. So I think in the case of like if you think of a world if you think of the world as purely math and coding and this is all gradable right and wrong then the RL thing is interesting right? like then the AI can get better at the thing because it's so objective, right? We know what right looks like, we know what wrong looks like, and then maybe the AI can hill climb towards right. But then in the spaces that are like messy and subjective, artistic, creative, you know, very much more human, it's where it's where the AI kind of just like, you know, it's kind of like dead in the
30:39water at a certain point, right? And um and then that's when we have like, you know, what we're doing. So like Joe will Joel will have it do eight hours of research of what humans have found to be good design over the last 90 years and then Jonathan will have it you know we'll have an image model generate 200 images before he picks one 200 names for a magical object before he thinks one fits the criteria right and so actually that is like it's like that's the it's
31:08everything all the 199 images that you and I never see represents the the the the taste that Jonathan brings to the table. You know, he decided to only show us one of the 200 and that means that like that curation is actually where his wisdom is coming into play, the creative wisdom, his his his personal experience.
31:29Um, right. And then Katie Katie was talking about how we layer, you know, taste is layered on top of story. And then I was realizing like especially now that I'm starting to make longer form videos, longer form conversational content through AI. I'm realizing wow the you know I can generate anything visually. We can make it beautiful. We can make it look like anything. We can make it look like science fiction but will it feel like a compelling story?
31:53And I'm realizing okay actually the writing skill set the pacing you know this the character design the devel that is the skill that the AI is not like delivering out of the box right and that requires the person the director to kind of infuse it with their vision for what what a good story is what a good charact what an interesting character looks like and so and and then it's like what what do we do to develop taste was I think Joe again had the most interesting take here which was that you
32:22have to consume consume a lot. You have to you have to see a lot of anything to understand what good even looks like, right? Um and so there's the only way to train it is to be out there and experiencing the world.
32:34Yeah. And by the way, I think you know I think it was not just here but an earlier episode is like you know writer skill was the most valuable thing you were undervaluing. Yeah. And and I do think like these things still don't like they can write a Wikipedia entry which by the way is a group collection effort that's not particularly edgy, beautiful, etc. It could be very informative. They can do that very fast and superhumanly fast and thorough and everything else, but like writing like interesting
33:03stories and and edge and dialogue and like for example I actually saw an investment memo uh yesterday that was like okay so which did you use chat GBD or Claude for this?
33:17It was basically like like like like even though it was a lot of like work that approximized what a human analyst could do, there were errors or softnesses or in you like it kind of was like it was like I was filling out a form versus it's like checking the box. Yep.
33:32Right. and and that's there and actually one of the architects um that I'm aware of in Japan actually has uh I think it's uh Chad GBT and um like produce 50 images in his style and then picks the three that he thinks would fit for a project and that's how he goes into a first meeting to say project and he's a super famous architect I can't name him because I you know I haven't got permission but like that kind of
34:01thing is still involving the taste, you know, as it as it plays. And it's one of the things I think will persist for some time at least. And it's part of the the you know, what is the future of work and how do we do it?
34:14It's one of the thing one of the many different areas to be looking at how do how do we um bring in essential and useful things as humans into working with AIS for you know high quality token output.
34:28Yeah. I mean, you got to think about the the chopping block floor and everything that never made it to to the public. And that that is that's a huge part of of of uh you know, what stands out. There's the slop and then there's the curated like artistic choice, right? So, we had Jonathan Brezzo, one of the most creative people I've ever met, honestly.
34:47And he brought a very interesting framework to the table, which was that, you know, creativity needs a human. And he had this framework of like the cog there are cog jobs and spark jobs where cog jobs are this like execution logic not necessarily the most creative stuff and then there's the spark jobs which is the design the music the creative choice the like the the the subjective space of things I think for me it's objective
35:16versus subjective where it's like if it's ex execution versus like more of an exploratory choice uh a subjective space I thought that was a very interesting the cog versus the spark jobs and what kind of tasks are cog tasks versus what kind of tasks are are spark tasks and he thinks that we're going to move people human beings are going to move to a place where more of us are going to be in this spark jobs kind of role working with AI um and I tend to I think I tend to agree you know
35:45it's a he said he said true true art is improbable right and this this really landed because I think about like what is the language model doing? You know, it's predicting the most likely next token. Well, it I once asked a language model to generate a thou, you know, a a joke every minute for a week. And then I looked at all the jokes and there was only one thing I realized like every single joke was a dad joke. And then I was like, why is that? Why is every single joke joke
36:14corny? I was like, oh, because the most likely punchline is the dad joke. The most likely punch line is not the funniest punch line. It's not the the this, you know, it's it's not surprising. It can't surprise because it's trying to do the predictable. And then it makes me think, okay, well, they're not and and if you ask the language model, if you go to chatbt and you say, pick a random number between 1 and 10, more than half the time you're going to get the number seven.
36:39And and then it's like, oh wow, like it's not even random because it is trying to predict the most likely number that a person would answer with when asked, pick a random number between 1 and 10. So you can't so like you know the right answer would be for it to roll a dice and then to say what the dice um revealed but then it's like we're working with these tools that are predicting they exist in the predictable space the interpol they're interpolating between what we have and then that's where it's very like the more interesting weird people that we have
37:08talking to these systems they're pushing it out to that that that like out of distribution like out of the predictable space and so I think about like that's where this that's how I kind of think about the Spark thing. Um do you think that do you think that this is like a feature of only today's systems?
37:28Great question. Look, I think um and you know you and I both know that being overly deterministic about AI won't be able to ever get to here. It only does is is is even though there there will be almost certainly some things in that to state it with some you know probability 100% determinist or probably 90% determinist is is a little bit of a fool's errand but I do think precisely because of the
37:56general ways that they operate there is the most predictive to token as you're talking about there is the with the mixture of experts and the kind of the learning thing the the learning paradigm tends to be the what is what is a high quality high prediction to this [clears throat] and if you're kind of being vanilla on the prompt then you pro the prediction is you want something generic or vanilla
38:26on the output we can massage this by being much less vanilla on the prompt that's part of what we're trying to help people understand to do in tokens the future you know podcast and so forth but it's also So um you know you put in uh workflow you know you have you know agents that play different roles that that that are then prompt off each other in order to make stuff happen including like red teaming or make that better or is that good enough and all the rest so you can improve it along those vectors
38:54like novel remixes of the way we attack a problem or create something.
38:59Yeah. Right. So, so we're already trying to like push the the limits of this, but I do think that the notion of um like how do we orchest this is part of the reason why we're talking about taste is how do we orchestrate to something that's that's really new that is improbable, you know, in the kind of the you know, Jonathan uh uh you know, kind
39:27of art uh and creativity side. I think that there is a much more enduring role for us than than you know and and and and by the way even as we kind of architect the kind of agents to try to do it. It's like one of the things that kind of lived experience and being you know kind of growing up in the world and so forth kind of helps us with. Plus I think in addition to that kind of taste and judgment is the context awareness. I would generally speaking think that
39:56we've that that that will persist much much longer than than you know kind of um the kind of AI maximalists uh will think that it does and it might persist you know you know air quotes forever forever. [laughter]
40:16Yeah. Yeah. I mean, it's it's like the AI is just chasing the weirdest of us, but like we're the frontier. Like, I'm in the real world. I'm constructing an interesting life and then exploring. And then the things that I would do, it can't because it's it's it's like reading a it's like the thing that someone once told me that these models, it's like they sat in a library and read every book. Um, but they never actually ventured out into the world. I mean, eventually they will and then they're they're gonna be learning from the world, but right now it's like they have
40:44this theoretical map of like how everything works and yeah, 100%. And let me kind of add a kind of a call it a a a you know um a vision, not really a hope or an aspiration or anything else, but not really a full theory, but it's like look, you want to combine the superpowers of these AI agents with our superpowers. And the theory that we have no superpowers is an interestingly articulate theory because like part of
41:13it is like well we see these new amazing superpowers from machine that we used to value uniquely in ourselves. you know, ability to to reason in symbols, uh the ability to, you know, operate in cognitive tokens and and you go, "Oh, it has all that." Well, is there any rule for us? And I actually think that um the likelihood that there's some areas that we actually in fact are much better um in collaboration, let's just use a frame of production, you know, com working
41:43with it is high, you know, can be lensed a couple ways. One we one one we said is ta t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t t taste, one we set is you know kind of context awareness and judgment. But like another one is like what most people track is in a in a a watt expenditure per token.
42:04Compared to these AIs. Yeah. Now the good news is for using AI, we can put terowatts behind the AI and we only have 20 watts here.
42:12Yeah. But to say, hey, look, that's not just a cheapness and efficiency of of token. There's also something about the way we do it that will likely have some useful advantages in, you know, kind of collaboration. And obviously we want to be the um the kind of human in the center, you know, kind of producers of kind of value in this, but like using AI to amplify
42:40our humanity as much as we can.
42:43Possible is produced by Pallet Media.
42:45It's hosted by Arif Finger and me, Reed Hoffman. Our showrunner is Sha Young. Possible is produced by Tanasiados, Katie Sanders, Spencer Stramore, Emozu, Amansuri, Danny Garrison, Trent Barbosza, and Tafoda Neimarunway. Special thanks to Syria Yalamanchili, Saya Sabva, Ian Alice, Greg Biato, Parth Patil, and Ben Relis.
