0:06okay we're on hi joh everybody and special guest haml Hussein hello ham hello hello sure most people know haml already uh famous language modeling at Al expert um and also famous teacher your um uh course last year on language models and stuff was pretty huge wasn't it h how how many lessons did you end up
0:36having you had a lot of content I don't know I don't know like 30 lessons or something it was quite a lot yeah thanks to all of you um you know who came and were there so I was just the facilitator really it's nice to have you join us um you do a lot of uh uh work helping helping some of the
1:04top companies in the San Francisco Bay area and Silicon Valley with their language modeling stuff including some of the top names in language modeling itself you know in the process you have to do a lot of writing and in your training of course you have to do a lot of writing and I think um this today's lesson is going to be all about writing so we were Keen to leverage your
1:30expertise so thank you for that awesome than happy to be here um before we do um so i' actually got slides today for a change because there's so much to keep track of I needed something to help my brain um got some new features and Sol it this week um uh the main one I wanted to mention
1:57is that you can now uh up to 128,000 tokens um are going to be passed to the language model now we've also made the amount of output created by a cell increased that to um 64,000 characters before it's truncated um we're doing caching more aggressively um uh to allow this to continue to be fast uh so hopefully this really opens up the
2:24amount of stuff that you can do um we this is only specifically for using the prompt option to ask Claude for help um the ghost Tech stuff is still more limited in the amount of context that we're using but we're working on improving that as well uh as we'll say later today we've also got some really interesting new approaches to um uh kind of adding context to the dialogues which we'll be
2:54showing you um and this will be part of our kind of beginning to explore retrieval that we'll really dig into in detail next week um but I think one of the things we ke to do always interested in doing is uh sharing cool stuff created by you folks you students um and I think jonno you had something some ideas you wanted to show there sure um yeah so I was pulling a
3:22few examples from the um from the Discord so this one was by um Allison I believe was the person who did this um and so they took the kind of embedding and uh databases and so on ideas from some of the past few lessons um Jimmy what's a topic that we talked about early on um using the aocd um package to access Adent of
3:52code so you can submit a query um oops and provided you've actually run uh run the code hang on where is this meant to be defined I don't see anywhere uh oh right so this isode they shared um I should have expanded that um some code they shared on the Discord um and it'll go in query and find as as long as it's not running in a live demo apparently it'll go query and find the relative time stamp in a YouTube in the Youtube video
4:21um so you can see the little ant equals 1,300 seconds that's like pointing to the uh the relevant bit there um nice not sure why that's freezing but um using tiny bird.co which is kind of a like a super based type thing I guess a um online database service um yeah I mean oh something just happened I was going to show some of the other projects I just going to say so given that it's using both geni from
4:50Google and that service if either of those are slow or down and nowadays with all the popularity of this kind of stuff these things are often a bit likey that could cause your demo issues yeah um so then a few other people had posted blogs inspired by some of the monster UI stuff that Isaac and that you've been showing so Daniel had one um Eric had one um and then while I was scrolling up to look for other examples I found um a diter
5:19had actually built a an example Gallery so they had gone through and found a lot of different um student projects um some more serious than others and um started pulling out the um I think it's just the top few ones that had easy visuals but compiling some sort of gallery of things that people are built to are all part of the course wow that's so cool so really fun to see like to that I guess on the Discord yeah yeah um I'm surprised didn't put only Scotch at the top one I'd be highlighting that but other than
5:48that it looks like a good project yeah yeah pretty great it but truly it actually needs like all 20 versions right and then it can get to the other projects so maybe a V2 um fantastic thanks I recognize Eric G Gaston from MBD I do absolutely
6:10um so okay so today we're going to be talking about um writing using the solv way um specifically writing Pros or Pros like things uh we've spent a lot of time talking about writing code um a lot of the approach you know so so solve it with code you know comes back to Po's how to solve it book in
6:39some ways a lot of inspiration there obviously it's not specifically at all about using AI although we are often leaning into using AI to get help the part of it and certainly we've been looking into using the AI in the Sol platform to help us with parts of our writing process um and yeah during the week jonno made clear on our Discord his uh level of
7:06enthusiasm for that kind of writing help um tell us what you really think okay um yeah well this is an excerpt um or a paraphrase from me trying to write down some thoughts after I tried to adopt some of the techniques that we'll be showing here and also just looking at like AI writing in general and why I personally find that such an anti- signal um so I
7:35don't know Jer do you want me to show some examples of like the kinds of AI stuff that has historically triggered my reactions and then my problems with it um and and then I think that'll frame the the way we've agreed to do this lesson so I said that on the Discord but I'm still here helping teach the lesson I'm going to be the kind of Grouchy antagonist in some of the discussions and then Jeremy and haml will probably try and sell me on some of the ways that they are overcoming a few at least of my
8:02objections um okay so one resource um this is a video from a Creator I admire um generative AI is parasitic cancer um and the illustrative example that prompted this video was searching for something to do with uh file formats for sharing 3D data on the web and specifically she found that pretty much every single result in all of the major search engines was either AI generated or templated
8:30uh content um completely devoid of any information right um just trying to promote some tool or other so here's a a file conversion tool they've gone for every possible input and output file format they've generated little paragraphs of it's very easy to convert such and such to such and such using our special online tool right and so anyone attempting to find the the actual information it's completely lost in this wall of um slop and slop is a technical term of art um I think Simon Willison
8:58defines it as AI content that is both un um askable and un audited like no no one's reviewed it or improved it it's just put out their whole sale right and this is annoying for developers it also has some negative consequences so various people have seen a proliferation of AI generated um mushroom Gathering books right completely hallucinated facts and uh AI generated pictures and some real downsides of it so this this
9:27is kind of like the one extreme of AI ated content right there's no there's no novel information which is one of my objections anything where you're not adding anything to the conversation that's a huge no no and of course like the writing that we'll be showing today I don't think any of it's going to be in that category um then like moving along the direction of usefulness um I've been playing with this deep research tool that open air recently released um and so this is more like okay you you give
9:56it a directive it goes and searches um I had some questions about growing crystals this requires the $200 a month subscription I believe at this stage I want to describe a little bit about what deep research is because I know a lot of folks in the audience are pretty interested in AI as a Tey yeah sure so this is this is a really interesting Direction and it's a tool that's designed specifically for this task it's a it's a quote unquote
10:25agent you give it a question and then it comes back with some clarifying questions usually and then it goes off and instead of immediately producing an answer it spends in many cases 5 10 minutes querying dozens or hundreds of sources and in between doing some some reasoning right producing a lot of thinking tokens to itself okay this is answering some part of the user's question but I still need to find out it'll go do another search or find some other results okay it's got that piece now it needs to go
10:55check something else maybe it finds a reference there so it's able to do very long Horizon multi-step actions to build up a huge amount of context and then distill all of that into some sort of we are somewhat bullish I would say at anai about this combination of reasoning plus agents um we're we've been pretty anti the agent hype in how it's been produced it's like no more need for softare engineers or write the whole thing for
11:25you just use Devon whatever which hasn't been true but the reason this combination is interesting is because um open AI here and also Google has their own one also called Deep research um has specifically trained a
11:45model to go through a sequence of steps to complete a task where in this case the task is go and find information about a thing yeah um so I don't know about you i' personally still like some way that a human was more in the loop and staring it and doing it iteratively just like we've been talking about but the results are pretty astonishing and like in our trials I've definitely found some pieces of content that have come back that have been interesting and and useful to me
12:13and that I've even shared you know someone asked me about a topic and I was like well I just happened to have generated this report on it um here I can share it with you um so it feels like that's I'm still not adding much beyond the initial Direction but the quality and the usefulness is High um and the synthesis is adding enough value that it's like maybe I could see publishing this being some sort of vaguely net positive contribution to the commons um but it's still it still doesn't feel like anything that I've had much of a hand in producing and so
12:43that's my other critique apart from just the you're not really adding much value is this this isn't anything like my voice or anything personal from me it's just you know this kind of like semi generic thing that's produced out sample that we're just looking at today um um which is um about uh so one of the things we'll be looking at uh next next week uh next
13:13lesson I should say is um kind of doing system administration and devops work with the help of solvent style approaches and one of the things we're looking at is openstack which is a very nice um system for controlling servers fets of servers and they've got a nice um python uh SDK um and before deep research came out we were
13:41exploring how to kind of scrape this for the purpose of providing context for a language model and weren't having much luck actually um and so literally just um couple of hours ago I just I said just do it for me create a map of how to use openstack from python what documentation pages should people refer to start here
14:08you know and it says like okay um you know just let me what do I focus on um and then it wrote a large um now the interesting thing here for me is that I had done quite a lot of research on this already so you can see it's kind of got links to pages that it
14:37suggests reading um and so I did a quick Apple F typed python um and actually I think I looked for specifically SDK and I was like okay right open stack python SDK yes that's correct that's that's what you should do yes that's the library you're meant to use uh yes that's how you install
15:04it um couple of little Snippets here um and then there was this uh um section about um fact that it's unified so you don't need in the past you used separate ones it's like oh that's like it actually yeah previously I had learned that I mean to be honest um uh pretty quickly
15:34um by using Sol it I hadn't really managed to do it by doing some quick Googling and so even reading their documentation I was like really confused about why they had so many different libraries and links and stuff that was interesting to me that in this case the reasoning agent compiled yeah pretty much the right exactly the information that I found useful had already found useful
16:08um yeah anything else Jon you wanted to um you know any challenges you want to set us here or um well so I I think yeah just to summarize my my like my two main things was either it's not adding much or it's it's um not in any sort of personal voice the thing that we did realize Iz when we were then like chatting about why is that I kind of have a an unfair and unusual position that I can say that from because a lot of the text that I
16:36write is either like showand tell of things I've done where the doing and the showing is the hard part and then the telling is pretty easy once you've done that um and a lot of the things that I never ever have to write are the types of things that most people do which is a lot of more like formulaic or templated you know um formal invitations or uh contracts you know there's there's a lot of types text that people do have to write that doesn't need an amazing creative Voice or much new information it just needs to follow some specific
17:05structure to say I do formally invite so and so to visit my house between the state and the state so they can get their visa so I realized it's kind of privileged in a way to say look the longest email I've written in the last six months is probably a paragraph um I I don't I don't have the same kind of need for a lot of the niches where AI assisted writing for specific structures and so on would be really really useful so it does help that has no management layer no HR no policy I can just send
17:35the bullet points right I can I can anyone at answer very happy to get the the summary rather than the expanded fluffy email so I really appreciate that um okay so
17:54um um so let me just you know I think think ham and I will try and show some examples of um a more solvet style process for writing which has been at least somewhat successful um so the first one jonno which I did explicitly as an example for this and I I I had a good way to test it
18:22out which was to summarize lesson eight which was the conversation Eric remember um and um here it is and so this is this is like a concise somewhat a kind of opinionated summary uh is this this one this is the longer one I think this is the longer one so I've got two versions I got that
18:52longer one and then ah yes and then I've got the more much more concise version um so I sent them to Eric I said Eric does this does this read well a and accurately represent what you're trying to express and he's like yes both absolutely he's like this is much better than other AI written stuff I've seen
19:21um uh I thought this uh little catch was actually great um I liked that quite a lot um the the the concise one I think did a really good job of like for me capturing some of the key ideas um like this uh this idea of the gardener you know um it's kind of like yeah cultivated
19:49Readiness um figure it out as we go with this idea of like connections to lots of previous things I liked um but I also like this idea that actually this is a pretty big deal you know um I'm trying to capture the key ideas I thought it did a really good job you know and then this kind of thing's useful for me because I quite often have conversations which I want
20:16to turn into a written form because of a lot of people prefer the more concise written approach um and then the longer one is actually more signicant this one is not a story of describing the output of what was said but it actually describes what was said step by step Jeremy said this jonno said this Eric described it this
20:43way now you might think well okay why not just publish the transcript but I'm sure a lot of you have notice transcripts are almost unreadable I I find them just like yeah you know we're not really great at reading a text version of what's said and it just reads badly um and so I've had a lot of success in the past with creating written versions of interviews and
21:12there's actually a lot of professional writers in kind of corporate settings who do this full-time um and I've used this with AI quite a lot at answer AI to to kind of yeah get something in writing for the te they say look here's the or here's the you know um the text version which hope you know which and I've gotten through it carefully and make sure it covers everything so
21:42um yeah this is different I guess to what jonno was describing where you were saying you you know tried this new library or this new deep learning technique and here's a picture I made and here's the code I used whatever it's like couple of hours of conversation and trying to turn it into something that hopefully people can understand in less than two hours uh so how about I show
22:10the process I use to create this Pho yeah sounds good okay so I'm not to be honest I'm not even sure whether or not I would publish this you know like at the moment this is a private tweet sorry a private gist um like I don't know if it quite reaches that level or not it might it's not that far away but it's certainly at a point where I would say to students of the
22:37course and I did put it on Discord and said like hey this is a good reminder if you wanted to review what was said or if you don't want to watch the video but quick version this you know as a participant in this interview I'm happy to say like yes this correctly represents the things we talked about um so um I used solve it for this process and what I had was a YouTube
23:05link um so um to download a YouTube link uh solve it will tell you if you don't already know that ytdl is kind of the preferred way to do it um
23:25so um so as per usual I used my template so I started with my template duplicated it you know which had the stuff that I tend to use um and then we used the Gen for Gemini again just like we have in the past um and um yeah I I asked it like okay how do I go about downloading something and it said you know something like this and so
23:53here's a useful tip is do this with the question mark and you will get the documentation of that thing um and it's I tend not to even bother reading it necessarily in detail anymore because it's just as successful just to have it there and now Sol it knows what you know what the
24:23names of the different things are and what what's there so it won't make stuff up I could have copied and pasted that into the context window of course but this is just such a convenient way to do it so it told me what to use and then I just told it here's all the details about how to use it and then I said okay I want to download the audio only of this interview and so it can see the documentation so both that I provided here and then also what I did do is I copied and pasted from there read
24:52me um yeah they've got a section in their readme showing how to use it so it's not very long right but I just copied and pasted and the readme is already in markdown so it's nice and easy um and then um while I'm here I'll just show you we're going to be using FFM Peg later so I also showed did the same thing grabbing the example section from the readme for the
25:19fmeg python Library yeah okay so told me what to do and I was like okay that's fine and you can also do it with the CLI it's like how do you do it with the CLI it's like okay that's fine too um and then when I tried to do it I got an error saying like oh you need to be logged in so um you know I didn't need to really discuss that with solvit I I know about
25:48that already and also if you Google it it's comes up pretty obviously which is that you need a cookies file and cookies is discussed in some detail in the documentation there's various options for it and these are the things which basically you know when you log into Google it stores your login information um so kind of common way to do that is with cookies from browser which will grab the cookies from your browser
26:18doesn't work in solvit though because solvit doesn't have access to your YouTube browser cookies um so instead you have to use the cookies file um and so it was interesting because there's not really like I was kind of searching around and nobody's got a great way to do that so I thought okay I'll create one so what I did
26:47find um is they've kind of got a fact you know um which talks about some of this stuff and they've got thiso approach that they suggest using which is you can um you can even create a separate user for this you know um and then you can basically create an incognito window you can go to
27:13YouTube um then you can log in um then you can bring up with F12 um the the console click on the network Tab and then if you refresh and then scroll to the top you'll see everything that it's downloaded right the first one here is YouTube and there's a really neat trick actually if you want to ever run on a server or
27:41anything like that a a download or a link or a button or whatever that's got all the right headers cookies everything you can always just say copy as K and it gives you exactly the right information that you need and the nice thing here is then if you do it in a incognito window and then close it those cookies are never going to become like stale whatever or not for a year or something which is the maximum amount of
28:10time so we just have to be able to pass those to um by TDP um and so it you know in the fact and stuff there's plenty of information about the general approach of what you have to try to do there um and I realized there's no particularly convenient way you have to put it in What's called the Netscape cookies format um and so I made something for
28:37doing that um which is a curl to cookies. text converter so I just paste my curl command in here and then I click convert and it downloads my cookies file
29:04um so I did that and then I uploaded that to Sol it um and was able to say-- cookies cookies.
29:18text and it worked now um how did I go about doing that um
29:35I actually used Claude And this is a great example of not needing any kind of iterative solvent thing this is a single step approach um and I just said create a single Standalone single page Standalone HTML artifact that does what I want I gave it the example you know um
30:06and it did as you see you know so this is the code and there's the output and then you know I looked at the code and I was like oh it's hardcoding YouTube that's a bit hacky so then I said like oh don't you know don't hard code YouTube but why don't you instead get it from the K command and um it made that change which is cool um
30:34so that that's part of this formatted template bit um still looks the same um so then um I kind of thought well be nice to have a really good place I can put all these quick and dirty little apps this is a this literally a single HTML file app it doesn't need a server or anything um so I created this uh little
31:02repo called apps and I put my first app in there and as you can see it's just pasting in the HTML file and this none of this required me to even open a terminal right so I downloaded the Claud one CLA artifact using the download button and then I just clicked upload um and then to create my um you know my little homepage here I then went uh add file create new file
31:31and just typed in um some markdown um and then I headed over to pages and clicked deploy from a branch and Main and that was it um there's a nice little thing if you click this little settings button and then check this box here then that's going to put in a
31:59little link straight to your deployed website so um I think Jon I think it you who pointed out to me maybe I don't remember that Simon Willison has actually created something almost exactly the same um which is cool so I guess uh yeah nothing's new um uh so he's got all these little apps
32:28that he's built um and I had a look at how he did it you know so he's using PL artifacts as well and I noticed he's also using basically exactly the same thing read me file with bit of marown in it um but I think this kind of thing's nice right like I think I feel like jonno this is something I always think you're really good at
32:56is you see something thing you make something it's small it's good to like have some way to get something out first don't you think yeah yeah and I think maybe the trick is noticing when it's something like that right here's a tightly scoped task it's not the thing you wanted to learn about today you wanted to transform Eric's lesson but it is useful you pretty sure an AI can do it you can verify the output because you can see what format it should outp put in um it's like yep that's the like
33:24perfect storm for having an AI write something you know it's like kind of just in time software single little piece of software and then yeah the ease with which you can share an HTML file with some bundle JavaScript is pretty fantastic um so yeah I I have a tools page with a collection of mine mostly some similar like a macdown file or something in a GitHub Pages thing mine happens to be a fast HTML app that just looks for folders with static files
33:53in them but um the idea is very similar and actually some things still I I'll just share and get a Pages just on their own um and then um haml reminded me this week that we actually released something few years ago uh so if you look for Jeremy Howard fast template um for folks who are interested in um maybe if they've got friends and family who are kind of
34:22interested in creating blogs and they don't want to use the terminal they don't want to use any Ides or anything like that we've actually shown a complete system here pictures step by step to creating um a Blog using GitHub pages that can all be done in the browser um so okay cool so
34:46then um I now have successfully grabbed this uh um audio file um and it's now sitting in my solvet instance um I was kind of wanted to see how long it is and so um it told me how to do that with ffmpeg it's quite really nice this FFM Peg python Library L of lot of not a lot of people know about it but I think it's like quite a lot easier
35:16to use actually than the complicated CLI often they do the same thing you know um this is just a wrapper for the CLA um with LMS although Gemini can handle that much audio it gets really slow and also if your number of tokens is over I can't remember it's over 128,000 or 256,000 each token becomes twice as
35:44expensive um so uh and also it just takes a long time iteration Loop um to to do everything you know with that such a big fail so I spit it into six equal sized parts and again you can see here ffmp grab the audio file start at this point which is the duration over 6 * I and then output it to this place copying the audio across overwrite something if
36:13it's there you know blah blah blah and again it's nice you know it's it's got some of my context it knows I like L stuff like that um yeah so that was my process for getting the audio file split up into workable size pieces um and now that I've got this it's really nice because now the entirety of YouTube is you know at my disposal to to analyze with a computer which I which I really
36:41like so in the future I can just duplicate that or I could pull some of that out into a function and you know import it um uh so that's that's where I stopped with that one and then I moved across to a new dialogue because I kind of wanted to have the audio bit separate to make it a bit easier to reuse um and then I kind of just to help um um the the AI you know I kind of printed out the files that I have and
37:10how big they are and then I copied and pasted from the Google Quick Start has a section about uploading files um creating a cached file um and then asking so this was all just copied from their quick start and again you know I could have put that into my context and asked solve it to make that but it's I think it's faster just to copy and paste from the quick
37:39start actually um and again you know something that haml might have mentioned once or twice in his life look at the data so I said please transcribe the Audio I really like your prompt because it's like the pragmatic diarization what's diarization HL it's like a um identification of speakers like who is where the sound is coming
38:08from yeah yeah and so that can be complicated normally it can be super complicated so it can be easy let's talk about diarization a bit so something that you'll be talking about later haml I think is circle back Circle back is something for recording and transcribing and summarizing zoom meetings and for that one diarization is easy because the zoom API provides for every bit of audio it t you know has the zoom username of
38:37the person next to it so that's going to be correct um for Mac whisper um um it's not quite as good but it's also not bad which is if I go record app audio um then you say system audio and turn the microphone on
39:06um it records two parts you know me and everybody everybody else which is not bad but in this case I had a YouTube video which has no obvious way to see who's talking but yeah it's really cool because um you know Gemini can hear the different people and so I said look you know I I was quite careful with this prompt it's like I wanted to
39:34know what answer AI is you know I want it to recognize these product names um I want to give it tips to help it recognize who's who um and uh yeah it's uh figured it out right I mean I said hello Eric and then the next person said my pleasure that's obviously Eric and it's got enough pieces that it can fit together here yeah so that's a good
40:04trick yeah so I was pretty happy with the way that was looking and now I'm going to show you something um super fun which is this is my text contains this text this the five first 500 characters it's got about whatever it was uh 10 minutes worth um of transcript and I want um solver you
40:33know or Claude [Music] to summarize it um and so here's the trick I say uh right in the form of a fast a St post a blog post about the transcription in dollar backtick text backtick um this is a new special syntax in solv it I say new we've been
41:02using it internally for probably good six months now um this is the thing I've been telling you all about that we weren't originally going to show you and we weren't originally going to bring to solve it but you know partly because the students in the course have turned out to be like way more awesome than even we expected um and also partly because we realized this has actually been fundamental to Our Success over the past few months as a company we should show you um it might seem like
41:30a small thing and it is but it's basically allowed us [Music] to incorporate a kind of solvet style retrieval into everything uh we do and you're going to be seeing it here um so let me describe what actually happens um behind the scenes you know the dollar sign here gets removed so that's not going to be sent to Claude what claude's going to see is it's going to see back itic text backtick and that
41:58it's also going to be passed a separate section in the dialogue which defines the variables it basically say these are the these are the definitions of the variables referred to in the following dialogue and it'll say text and then it will have it doesn't have to be um text it can also be dictionary uh in which case it be stringified I think could be an image um um uh some things Claude doesn't know
42:26how to work so Claude can't do audio so what it doesn't work for everything um and you can see what then happened is it it did it um so why is that simple feature so powerful um I might need your help jonin Hamill to describe this because you've been using it a lot as well but basically it's
42:54like it kind of closes the loop we have code we have the AI the AI can see the write but it can't see the contents of the symbols we make and so now you can provide that you know and a lot of things automatically pass like the whole set of symbols and that doesn't really work because you could have like a 2 gigabyte video file or Panda's data frame of a genome or
43:23something but this way it's again it's this kind of human in the loop thing we're deciding we want to tell solve it about this thing now give it a name I don't know it's got nothing to add it yeah just just that it's it's in the same way I think some other systems have something like this right where you can tag specific documents or something um we we started working on a magical it retrieves the right things behind the
43:53scenes version and then we realize that actually just explicitly saying look it especially when the the syntax is so simple like a dollar sign and then the backtick and then the name and then the backtick like and that thing in backx can be you just loaded the transcript of a video or you've loaded some content from a web page or we'll show some other options as well it's like okay I have I have a thing I have a way to refer to that thing I have a way to preview that thing and I have a way to tell it to the language model this is the thing that I'm referring to and have it share that
44:21context with me um it feels just like what you do with a human right you'd say okay here's my main conversation with you can you do this and also I've attached the email that I want us to talk about um it just feels like that same flow like here's what you need to do the job well yeah something that some of the big companies are now providing also the ability to have a model watch your screen while you're
44:49doing something which has a similar Beauty to it and we've definitely seen some great results of folks stepping through complex devops processes that kind of require using the browser and again it's like having an expert human there because they're seeing the screen they can say like no not that button the one slightly underneath it idiot I said the red button don't press that one um yeah so
45:19it's been I really yeah I mean to make a comment also is it's the it's a I prefer doing a lot of the work that I was previously doing in Claud projects um you know now in solve it because of this syntax because it makes everything a lot more portable you know I can sort of give something to jonno and I'm not overwhelming jono with like
45:48all kinds of attachments and all kinds of stuff they're programmatically brought in and then it's easier for someone to edit what it is and it's a lot more readable um You Know Than This Big Blob of like lots of text lots of attachments so that's one re another one thing I like another thing I I think is great is the prompt makes more sense like a lot of times with Claud projects or whatever the whole
46:16prompt sorry just before you continue h there's bit bit of confusion in the chat just want I explain again for folks who missed it this is not a string interpolation right what happens in the dialog is the dollar sign is removed so the language model just sees back tick text back tick and we add a special section to the dialogue saying basically to flaud here are the definitions of the variables being referred to
46:44below and text is a symbol which is to say a variable right and there's all kinds of variables we support we're going to be adding more and more when we don't have at the moment for example is like what we'd like to add is if it was a function perhaps we could put the source code for the function there um but this is a variable containing some text so you don't have to save it to a file or anything so sorry Hamil just go on just want to make sure people were on the
47:13same page um think I forgot what I was gonna say oh no sorry no it's okay um uh no I think I forgot okay and then a question was like What if you had something like DF do head parentheses um no that's not going to work I don't think we make that work it's just we're just literally symbol it has to be a
47:40variable or a function name or whatever if it comes back HL you can tell me uh um so the process of getting AI to help with this is something I've done many times now over the last few months and there's a particular process I found that works well and so I'm going to describe the process
48:07so um ideally actually I would have included in the context sorry included maybe I would have read into from a file into a symbol uh into a variable the text of a previous summary which I liked and I would say do it in the in the similar style to the this for instance but Ham's going to be talking to us more about that approach um but this is assuming you don't have any examples of similar
48:35things so we're starting from scratch here um and I just asked it to do the first bit right this is just the first bit so I've said um so describe my audience right describe kind of get all the style I want you know ideally some kind of reference style so a person or an organiz that wres in the way that you want to be similar to um and here we have it right and then
49:03this has got basically almost no relationship whatsoever to my to the original version I rewrote nearly all of it um but it was very helpful to have the starting point to might to just let me know what was covered in that part okay so so there's a few things there Jeremy right like um okay so one I was objecting to like you're not adding anything here obviously you are because you've got this whole whole transcript that we're drawing from um but also there it's
49:33like you could have just written the first three paragraphs but but that's hard right like that's that's not easy have to relisten to that whole thing and right and also it's just intimidating to have like a blank thing so yeah yeah you know it it might seem odd to someone watching the person like wait you had it generate and then you completely rewrote its response M like a why not just write it yourself B why you editing what it said in this way but it's like oh no that's it's getting you past WR as Block
50:02it's getting you a first draft that makes you see what you actually want yeah there's a lot of nuance in this step that's worth like dwelling yeah there's a lot and and actually the payos comes later right which is then okay so I'm now going to do part two so I I've th thrown away the bits of the dialogue but I just said to solve it you know combine the previous things into a function run the function to get part two now just put the prompt into a
50:29variable um you know run it there's lots we could do to improve this so one thing that would have improved this a lot would would be to also um add the previous summary and transcript to to the thing asking for the content because that would help it know who the different people talking are and stuff but I was kind of doing this the quick and easy way and please continue
50:58following on from your previous response use the same style as before because now I it was actually my style you know um so then same thing right so I edited this it required less edits than before um okay keep going continue to use the same style okay this one
51:22um uh sorry um required less edit still uh oh that was two here are three less edit still um and then four same thing keep going and I didn't like this all these lists it lists lists I don't know sometimes this is the nature of Auto
51:51regressive models they can get into a certain mode you know there's a certain there's a couple of ways of kicking them out of the mode I could have gone back and changed the prompt but actually this is a big tip this is great when it goes off in a way that you can clearly describe keep it and then tell it right get a better version edit it until it's better still and now it's got an example of a
52:19negative and a positive example in its dialogue so then you specifically said yeah that's good go going in that tone and this actually gave me another negative example which is this is too short you know uh so I said like okay and doing this thing where it's like I don't know I wouldn't write like this download button Revelation it's like uh more detailed
52:49less hypy longer so this one um I don't think I changed it much at all at this point this one is pretty good um so then for six I'm not even sure I had to change it at all right and now here's another part of the key step you keep all that context and you go back and you do it again okay I think we've improved our
53:17writing redo the first one and I reread the first one and I read this one and it's way better so here's another trick is I again we gives it the feedback right and we ask it to describe the Improvement right and I I agreed with it I deleted a couple of things and I changed a couple of things so now it's got a third negative and positive
53:45example and a list of like specific guidance and then I added this you know because it said don't use ja I like no no it's okay to use J just explain it like okay now do redo two redo three and I didn't have to edit these at all um and that then I just um manually took those six markdown things by just clicking on the copy
54:13button and pasted them into a gist and I sent that to Eric and Eric's Eric at first was slightly skeptical he was like that sounds really good but are you sure we said all those things I like yeah I did each piece one at a time I checked them it's it's like it's really good so um I think that's a key step you know is to bring in an independent third party uh to check
54:41it's easy to get into a bit of a trap of when it starts doing its over hypy horrible way of talking that it often does is to not stop it early and try and get into this you know um kind of theme so but I like this like it it's something I really struggle with with these descriptions of an interview is I tend to just write Eric said this Jeremy
55:09said this Eric said this Jeremy replied this and by this point I really liked the way it's like you know in a discussion about tools Jeremy how shares us an experience he describes after doing this Jeremy decided to try actually this might have been J maybe you got The Voice wrong do you remember Jonny was this you or me talking about this shell scripts I don't remember I think it was you okay um and these kind of like segue bits
55:39between sections they're really important for good writing and it just did it for me so um you know quotes nice otherwise I would have had to have gone and tried to find quotes which is really boring so I think it's like this is one of those cases where like quite often for this kind of my audience is a dozen people you know it's the folks in our team um I want it to be good enough that they're going to read it but it doesn't have to be good enough that it's going to
56:09be you know um published for money or something um so I felt that it definitely achieved that level and then for the um uh concise version I actually just basically grabbed the whole dialogue and oneshotted it into R1 r1's a reasoning model and basically
56:37said you all create a concise summary that catches the key underlying idea um write it in the style of Jeremy Howard and I think in that case I did provide an example in the context of a of a post and um it was quite interesting it it really is very good um there were bits of it that were actually kind of inaccurate or kind of overreaching but they were easy enough to edit um because
57:06with the reasoning models they they're not so they're not great actually at all for dialogue for long discussions if the reasoning models go wrong you have to go back and reprompt them basically
57:25um okay so I don't know jno does that sound like okay you of AI helpers here definitely I I I think there's a lot of reasons why you could put your stamp of approval on on the final result there right you carefully edited it you approve this message you started out with a lot of iteration and also like I mean we've been talking about polia and solving code problems a lot this course
57:54and if you zoom out and you ignore more the subtleties of like the AI piece like what what were you just doing there is like okay you framing the problem right you starting out that dialogue like I want to create a blog post based on this transcript or you know you're like stating the problem in your own words you're like figureing out what you want to go for and then you're breaking it down into small pieces like let's do section one then section two then section three and you're checking them as you go right making sure there's a fast feedback loop that you can see something and then make a change or in this case like maybe prompt solve it less highp longer answer Etc see that
58:24change in action like validate that it's going in it does feel very much like some of the same methods that we've been going on and on on about and even like I have like this is an example the handling handlers section of fast HTML docs is written using this approach um with AI and
58:46like it it's kind of like it's it's in the reference section it's not meant to be a beautiful praic explanation of themes it's like I literally wanted to go through every damn feature you know um and previously the only documentation of of every damn feature for the tests okay so what I did for that was I was I um basically made a copy of all
59:13these um and then each one I just said you know to to to Claude uh to solve it um you know explain what this t is testing and why you know it the same deta deal like I edited the first examples like here it is right edited the first examples quite carefully but you know by the end it you know this I think was
59:42entirely written automatically and these models are pretty good at guessing this kind of stuff of like okay what is being shown in this test and why it's like okay well this is showing curly braces in the root it can see the difference between this test test the previous one which is its curly braces you know uh and it so and it knows oh that's a Starlet path parameter you know and it understands why you would have a name on a parameter um yeah this was and I swear
1:00:11I would have gone crazy if I had to do this all manually and I didn't right until until I had this system set up I didn't I hadn't documented this I just wrote the test is it fa to say the commonality between the two is uh or one important commonality is that you had some strong context to feed in in the in the solv it lesson you had the transcript and this fast HTML example you had the tests that's certainly part of it and
1:00:40another part of it is I was not aiming for magnificent Pros you know I was just aiming for good enough Pros yeah um I'm suddenly aware that our special guest is the first time we've had haml speaking for the the hour of the lesson so we should maybe um skip to his to actually okay sorry say I was I thought we had some less no worries you don't have to have just for my the sake
1:01:09of my voice um no that's where we're up to actually that's that's the segue so this so um the segue is yeah on the other hand um hammer wrote a couple of things where the quality of the pros mattered a lot um uh one was a post about um Devon um the AI agent uh on our blog which went to number one on Hacker News
1:01:38it was like people literally wrote about it as news of like haml said this um it was pretty huge wasn't it Hamill their response to it was big yeah I was just blown away by it honestly um another example it's you you know wrote this series of AI Essentials for Tech
1:02:04Executives um which has been so well received that O'Reilly are turning it into a book um like directly they test like you told me they just gave you back the first draft and they basically didn't change anything right yeah yeah so let me share a bit of an anec so so I was not involved in creating the axx thing but I was heavily involved in
1:02:33um creating the U the Devon thing um so we uh paid 500 bucks a month to get a Devon license for the company and um uh we were all pretty excited to give it a try and after a while we were all like oh this not great um and EM was like H I don't think it's worth paying for this anymore I like okay well that's that's that's interesting we tried this
1:03:02thing and it didn't work out we should write that up you're like okay I'll try and write it up and bit later you're like okay here's the pr for our blog you know you want to review it I felt bad because I was like as I literally said haml we can't post this not only can we not post this there aren't tweaks I can give you that we get it like you have to start again you have to throw about the whole thing and start
1:03:31again and and then you redid it using this completely different process and so tell us tell us about the process because it's a process that I don't know brings together a whole lot of the tricks that you've learned and and the result of that process was really really strong piece of writing um which a lot of people have reacted extremely positively too
1:04:01yeah so at the heart of the process I the fundamental thing to remember is I think a lot of writing uh a large class of writing is storytelling you know you want to tell a story and that relates back to your example they gave earlier of how you took the class with Eric and turned it into you know that post um or the transcript of that class turned into a
1:04:31post um I would all you know I would say you can almost invert that thinking and say a lot of times you know you want to you're trying to tell a story when you write and so one thing we found at answer AI is the one of the best ways to get that story telling started is to tell another human that Story in real life yeah and and and I knew about the power of this because a
1:05:00number of times you had interviewed me with no audience with the entire purpose of the audience being a language model that we then got to like pull out my stories and you're exceptionally good at doing that interviewing so that was that was the first thing I offered up was to say like okay Step One is let's do that thing but this time I'm going to interview you right because I knew you had lot of strong feelings about this you know you cared a lot about this
1:05:30topic yeah it's very interesting I've always been the one interviewing and then you offered to interview me I was like oh that's interesting okay and before you interviewed me I was like eh I don't think I have anything interesting to say or I don't even know like I'm not sure what I'm going to even say about this and because you know me you were able to trigger all the things that I know how to trigger you like
1:05:58strong emotions actually um those those emotions being like frustration anger disappointment whatever all of those things you're able to trigger all of those surprising to me honestly and um capture that in a like capture that conversation is is basically like very very strong grounding for a blog post because in a blog post
1:06:27you know the more personal it is the more sort of you know one thing I found is in in writing if you can make the introduction start off very personal and relatable um and have a strong some kind of strong emotion it does go a long way people resonate with that it's very you know yeah it's it's just something about that and we did a zoom interview
1:06:56so we just popped open a we just like let's do it right now popped open a zoom you added Circle back bot to the chat it autogenerated the transcript so part of this was there was no overhead to like organize and prepare and separate steps we just pretty much instantly you know went from let's do this to you had a diarised transcript of that discussion yeah and I can share my screen of the
1:07:25solv it uh sort of version great of this so this is once you had the transcript yeah um so okay so like you know based on this trans so like I had a prompt that looked like this um you know I had a old version of the blog post okay so you're using the dollar back tick trick yeah using the dollar back tick trick and I
1:07:52said um you know provide suggestions I can improve on my earlier post oh so you've got the earlier post somewhere earlier in the discussion uh yes um or I think I deleted that by accident sorry in this uh I was trying to clean it up for presentation yeah yeah and um oh yeah
1:08:17sure um and so like okay you know it gave me a summary um and then I said okay okay in addition to this Jeremy provided me following feedback and this was your candid feedback um you know that you gave to me personally and I said okay how would you and I kind of went step by step Here original blog thing yeah oh yeah here's sorry and then uh I said how would you strengthen the introduction in the
1:08:46original blog post based on the feedback I this don't write it yet is interesting I do that a lot as well it really wants to jump into writing but this gives you something that's easier to edit the output right yeah so that's what I did it's like hey don't write it yet tell me what you're going to do and then uh and these first steps are really crucial like you want to pay really close attention slow down think about is this really the plan you want
1:09:14it to carry out can you inject anything differently in here SL down constant piece of advice we hear students saying they have to remind themselves when doing Advent of code problems anything solve it it's like slow down yeah yeah um so okay like it gave me that I edited this I said okay great please rewrite the introduction I only wanted to rewrite the introduction um and here are some
1:09:42writing samples to ground your style and tone and I gave it three writing samples that I thought were that I kind of wanted to sort of that thought had some Rel to this post is this your writing or some else's writing just it doesn't matter this is um my writing this is some of your writing as well okay so this is critical right you're providing in context learning for the artificial intelligence model like this
1:10:10is more subtle and important than perhaps folks that aren't deeply familiar with AI would realize but the way these Auto regressive models work is that they're going to continue in the same style of writing that they've recently seen if you like so this is is like taking advantage of of how models work you know is to is to incorporate this this text here it's gonna it's G to keep writing like that
1:10:37yeah um because I knew of its plan and it was you know I thought about it I was like okay initial hype in context you know this is kind of from me yeah I knew that okay there's initial hyp in Contex like well let's be you know let's ground it in the actual initial hype would you be able to share this um it's just somebody's asked in the chat would you be able to share this after the class maybe as a yeah absolutely yeah absolutely
1:11:06um and so I said okay like here's the original Devon blog post by the dev team I gave it this link because I knew it wanted to include it in the post but that was you know and I said uh and like a lot of times AI likes to write in bullet points it does so this is a defensive mechanism do not overuse bullets some are okay um if I don't do some are okay then it doesn't do any bullets so I kind of have to give it
1:11:33like uh they're not g and then if I yeah oh I heard you like paper clips yeah uh and I did this interesting like if you think a particular picture and screenshot might be interesting at a certain point go ahead and put a placeholder with some kind of caption because I knew like I would like some visuals but I wasn't sure where I might put them and I just wanted to see see where a nice breakpoint would be um and then it gave me this
1:12:01introduction I had to so we'll say a couple of things about this one is I had to heavily edit the introduction a lot a lot of deletion happened um usually in the case with AI it tends to yeah I just have to delete a lot of stuff that's not necessary um you know I want I rewarded things to be less hypy um I had an image I have an image in
1:12:31here but you can't you know I don't I may have Ed I may have messed it up um you know I put the image that I wanted in there um and I just really carefully edit it edited it and I also spend make sure I have the title correct I spent a lot of time on the title it makes um it clarifies my own thinking about the vibe I'm going
1:12:57for and also find that tends to yeah like the the earlier text especially the title also like grounds the rest of the the generation but it's really the title the summary you know like you have a title and summary in a lot of blog post so get that really dialed in like think about how you want to communicate there's some principles of titles um but titles are really important you know when I I struggle a lot with this and
1:13:26this is where often like also I would do the whole thing twice so like after I finish I would then be like okay let's redo the first one suggest 10 titles now that we've been through all that process you know and then redo section one now you know like it's doing it twice can can help at least for me I often don't really know what I want to write about until I've Drive written about it yeah yeah it depends on your style like
1:13:55I try to have some title that captures some emotion or something that grabs me and then it once I have that then I can feel more comfortable writing so but some people don't feel that strongly they WR like the key thing here is if you can kind of curate a relationship with someone who knows you pretty well who you like have a bit of a conversation who kind of understands a bit about your thoughts about a topic
1:14:24and like work together on interviewing each other in this way you'll both get better at it you know so like I've known haml for a long time and I knew about the things that totally pissed him off I knew about the things that triggered him and I knew that if I could capture that in the interview they would end up in the transcript in front of his face and he'd have to be like okay I admit it I am pissed off about that and then he'd be like he can't say like oh I got nothing
1:14:54interesting to say you know so I really tried to ask the kinds of questions that I knew were going to generate the response that was going to be interesting and personal and I've always said to ham you know with writing is like all right dude start with something personal you know start with your experience and your feelings and that's what gets people interested and it's probably very different to what people learn
1:15:21particularly from an academic background yeah no definitely the interviewing is is really key here um which I think is is I haven't seen anyone else talk about in their process and I think somebody in the chat mentioned that um Renee Brown writes the first draft of her books by inviting a group of friends to her house
1:15:48and talking about it out loud so that's cool yeah that makes a lot of sense um okay so here you know just spend a lot of time on intro make sure it's really good um you know the exactly like way you want don't go beyond it unless you you know get this going on um then you know kind of iteratively like good Improvement upon the original introduction so it is kind of
1:16:17interesting because I did have a bad version first which I think helped a lot like in a way just like how Jeremy described earlier with the negative examples like it so maybe if I didn't do that first and had kind of bad example maybe this would end up being a little bit different flow who knows but um you know I'll say how would you improve the following section after the intro don't write anything just yet outline a plan of how you might proceed gave me a plan I edited the plan
1:16:47a bit not as much as I did the first kind of you know uh time but it seemed like to do a much better job um and I said and after I edited that I said okay go ahead and write this and great what is Devin um you know whatever did this I edit this I edited a little bit but not as much as the introduction um mostly
1:17:15deletion again um but not as much editing of in like correction I would say um I did have to Rel language some things but again less um and then I knew where I wanted to go next so I said okay WR write the transition into the detailed analysis of specific
1:17:38failures um so kind of did like again you were getting that like positive feedback loop that we're always looking for where yeah you were having to do less and less Yourself by this point I was having to do very little um and you know by the time I got here I was like wow okay this really feels like but I would write and this is the opposite of what most people say their experience with AI is that
1:18:08don't use this approach is that oh the AI gets sloppier and sloppier which is what's going to happen right if it starts out writing something a bit sloppy and then it's getting that in context learning of like writing slop it's going to get worse and worse with a few little pushes it's going to get better and better so it's quite a lot this time um and I found that it was pretty good like and then I knew where I wanted to go next so I can transition from this to sharing quotes um from jonno Isaac and
1:18:37haml so I wrote that that was fine and then I said build on the original conclusion bringing elements from the transcript giving the reader a strong sense of where we stand had to redo it it kind of had the problem with waffling too much so I said don't waffle um and then yeah I edit the conclusion I find that I kind of have to edit the conclusion too I have sometimes they terrible at conclusions I don't know why AI just yeah
1:19:06it's tying it up in a bow who wants to like doesn't like dramatic or either goes like to dramatic a conclusion or to like not really having an opinion about anything and know you know just I haven't figured that out but I think maybe if I gave it lots of examples of conclusions maybe it would do a good job yeah I recently found that someone on her team Ren did this experiment where he gave like because we have the same
1:19:34issue with titles like I haven't been able to get you know a show that actually yeah so perhaps if you gave it lots of examples of conclusions specifically then I'm I'm bullish that you could get much better conclusions like examples of blogs and con clusions I want to make sure we' got plenty of time to talk about your other one as well the ax one would it make sense to switch to that now or is there some more
1:20:04yeah we can talk about that um so um I guess I know from talking to you about it before it's it's it's it's actually pretty similar I know this one that one you co wrote with a friend Greg but let's ignore the Greg bits and focus on the U bits could you um and again you know the outcome widely read book people really like it are going to publish it it was
1:20:33originally just going to be an email newsletter but it's got bigger and bigger um so very successful as an as a piece of writing could you describe maybe first the similarities in the process and then maybe if there are differences you could touch on them as well yeah yeah so um and while while you're talking I'm going to share my screen just to go through the site so people can see it
1:21:02but stick not me yeah I think the similarities were there's definitely is extremely similar I wouldn't say it's as polished as the Devon post per se like I think I got better since then when using AI even um you know so the process here was I didn't have someone to interview me for this so I
1:21:31use um this app called voice pal which is a you know it's a voice transcription app you can have install on your phone and uh you speak into it about whatever and it asks you follow-up questions like yeah see on your screen you have an example um see is asking followup questions and sometimes when you don't when you have writer's block it's a
1:22:01super effective because it just get you know you you you're like oh okay I can respond to that it just gets the ideas flowing which is super helpful 5.0 review that's pretty impressive don't see that um I've been quite pleased with it and so yeah I just go on really long walks um you know walking is also a good way to think in general it is a lot of like
1:22:30cognitive science or some you know whatever behind that and um let's just talk about that briefly because maybe some people una aware of that so A couple of things going on there one is the walking physical process is actually pretty good as long as you're not going too fast um so I actually did a lot of analysis of this myself over some years using a walking desk um um and backing my productivity um so that helps as long as
1:22:58you don't go too fast but perhaps more importantly the there's a lot of research that shows um changing location is uh good for your creativity um which might sound a bit hokey but it's it's it's a solid result um because your brain responds to its environment to to generate ideas it's not like a random number generator so when you're walking you're getting both
1:23:27benefits and ideally yeah ideally try and walk different places each day as well um the when you're referring to the The Walking desk is the is one thing that's coming to your mind is learning Chinese yeah yeah exactly yeah yeah that's a really cool video usually everyone should look at that there's like some talks from Jeremy about him learning Chinese it'll blow your mind it's like me um uh but I really yeah I really admire
1:23:55the way you take advantage of this also when you're driving you know you have these ways to you know I guess just pop your airpod in or whatever and record a conversation while you're on the Move super smart oh yeah so um walking you know driving is kind of an activity where by default I can't really be doing anything productive uh normally so if I'm you know I have to drive somewhere it's
1:24:24going to take like an hour sometimes I'm like okay I have an idea for a blog post now I can't do the back and forth questions as much because I have to you know pay attention there's another app called super whisper um you know it's really good like it'll have on device models so you don't need internet connection and you know you can just I just record and just try to dump my thoughts and just be okay with the long periods of Silence where I
1:24:52don't know what to say um and I by way another trick that combines these two things is um call people while you're driving you know so sometimes I know Eric's going on a long trip when he calls me out of the blue he's like hi Jeremy like hi Eric what's up like um nothing much like okay where are you it's like Marin where are you going uh Stanford like okay so you got an hour to
1:25:21kill like yeah I got an hour to kill so you know and uh that's a great time to brainstorm ideas through a book or talk about getting feedback on a post or think about you know feedback on coding ideas or whatever that's an interesting idea the recorder on and that's Co what's a nice way to record like phone calls I have to explore that I don't know off hand those things have
1:25:49a recorder call you know call recorder thingy that you can use yeah yeah it seems like a wonderful idea you're always wel to call me HL if you want to company on your drive yeah uh might take you up on that one day um yeah so it's really I used that a lot and you know just dump my ideas into
1:26:18this transcripts and try to you know um sort of curate those a little bit you know because I would do segments of different transcripts and that was the grounding for that particular thing um as well as you know I gave it a bunch of blog posts that I wrote some podcast that I did um I try to think really carefully about the context I probably spent before I even
1:26:47you know I probably spent half an hour just thinking about context like after Gathering it even just okay what I wanted to sound sound like when I want to emulate it started as an email course I found other email courses that had good emails in them are good Snippets of knowledge um and I gave that as an example you know so I just try to curate SP a long time curating um by the way here's the for
1:27:16recording long as you're notan byin oh Europe living in Europe's getting worse and worse oh this is native wow I didn't know that I think it might be somewhat recent okay I need to try that um so yeah it was that and then basically going through this itative process it was a bit because I didn't have someone interview me I had to De I
1:27:44had to do a lot more editing I would say uh this time um but one thing you did tell me is that you did get that feedback going on right yeah eventually it took a bit longer um like the first two or two or three I had to edit quite heavily but then after that then it became fairly uh yeah it
1:28:12became you know better at following My Style and what I would say I feel like all of these things like the the trick is reducing that activation energy to go from like nothing at all to a bunch of ideas from a bunch of ideas to a title in an introduction from a title in an introduction to section one like I feel like what you're describing here and this is something you've mentioned to me
1:28:41quite a few times is the idea of like unblocking yourself yeah um there's a lot of little tiny ideas like you know yeah there's um sometimes I would just brainstorm so before even doing this I would I brainstorm within like a Claud project like oh what should I even talk about what should be the what should be like the outline of this thing does it even make sense um I didn't even you know just to
1:29:11get some ideas flowing get some things going um because I would get stuck and also I didn't start with the book If I thought started with the idea of a book I would have not even a AI or no AI probably would have not done it you know it's like oh let me just make an email course like no one's going to read an email course like you know that's what I thought and I'm like so low risk I just didn't do one email at a time that is like it's at least I find that too is
1:29:40like try to convince myself that no one's going to care very much about the thing I'm doing otherwise it's yeah too much of a wory to be like oh it has to be good yeah exactly so I mean this starting small and iterating one piece out of at a time you know one email at a time whatever um uh yeah because this is more business writing and not let's
1:30:08say less it's less storytelling there's certain storytelling going on there but I used I did try to simplif make sure the writing is like you know sixth grade level or below yeah um and actually this reminds me of something you're telling me about earlier before with a call which which was you were trying to you know you know you you were looking at getting AI help for writing a kind of a nuanced take on a rather complex topic
1:30:36which is like super different to this like AI basic rxx thing right which feels like maybe Nuance is still something that it can struggle with I guess yeah yeah it was a perhaps a badly chosen example like okay I want to try these techniques but if you're you're going to be saying something that could be interpreted as insulting somebody or picking a fight with some idea or some
1:31:04ideal but you're not but it's you know like there's still places where I think there's something to making sure I'm saying every word rather than having something that could you know go a little bit off the rails felt it felt too precious like oh this is going to like put words in my mouth that I didn't want to say or something like that so um yeah definitely like it seems like the subject matter also determines that point at which it starts
1:31:32getting really useful and how much you want to even involve it at all um I I don't think any of us want to be the like I wrote this hot f l it to my parents bying CH gbt you know um but I I gave my investor update sure but
1:31:50ideal actually I care about investor update so much I delayed it until Eric is visiting in person then we're going to spend a lot of time on it to make it really good let not use that example okay yeah sorry bad example bad um one more thing that might be interesting to bring up is um about unblocking yourself like another thing that I find that I hate creating is slides like it feels daunting at first like how many I'm going tell this like
1:32:19slides and um I could just share my screen um there's this app called gamma G MMA aapp so um I wanted to have like a funny presentation about eval because it can be a dry topic you know and um and basically a presentation about this AI exx thing but it's it's dry AI exx like who wants to listen to talk AI exx like
1:32:48make it entertaining so our idea was like okay like flip it on its head and just say how like how to have a strategy that fails and make it funny but I wasn't feeling in a funny mood I was like I don't even know where to begin how do I make it funny so asked this app to like make do that essentially and you know these slides are not great it's like okay you know um it's like meet your guys to failure blah blah blah and it
1:33:17was just put me in the mood of like being funny and then I could like actually be funny like I could say okay you know I could have slides that say like okay example drown in metrics or let others dictate your metrics use all of them or avoid looking at it just gave me that initial guidance like I needed Guidance just like a language model like in a way to be
1:33:45funny and Hil also like I think on a very different topic something I've not you do a lot is is you're pretty good at recognizing when a piece of writing is actually more transactional you know like oh you know they need a proposal or oh they need this security questionnaire filled out or whatever and you seem to have a pretty good process for
1:34:14like you know generating possibly not very high quality writing when you don't need high quality writing yeah so um I do Consulting and this I have a Claud project for my Consulting proposals I can probably move that over to solve it I probably will soon um so yeah I can show some of this like basically what we have I I give it a lot of examples of transcripts and proposal pairs that's
1:34:41fantastic yeah act actual in context learning examples yeah and I can show you my prompt every one you do is going to become another one that's going to improve your FL project yeah definitely yeah and I keep you know updating it so like basically this is this prompt you have the skill level of a partner at paler which is a consulting firm that roughly trying to draw inspiration from
1:35:08although this maybe not the best uh yeah it's something it's something to get started with like professional proposals um who's an expert at generative Ai and artificial intelligence selling Consulting projects um you know your drafting proposals on behalf of Hamill Hussein and it said you will receive a meeting transcript and or notes from which you must construct a proposal from and then I give it an inventory of the
1:35:36files like okay you have a transcript you have a proposal you have a transcript yeah the dollar backtick thing would work well for this yeah it would work really well um you know I just said use these examples to inform the correspondence between the raw transcript and the notes if asked provide multiple uh tiers or option options but know that this is only for the purpose of allowing the customer to compare the price to something else um this is kind of because there's another context in here called $100 million offers it's like a kind of a book
1:36:06on creating proposals we don't have to get into that right now um I said you know whatever I have a in so um I have some guidelines of like how to write get to the point short sentences repeat the customer's pain points follow the guidelines in proposal guidance I have proposal guidance up there so there's a bunch of context and I've got a really kind of minor meta tip here but um yeah please
1:36:33the these kinds of uh system prompt text boxes including this one they tend to be very small um and so they tend to encourage our easily misled brains into not writing much into them um so just something I think is a good idea is actually to open up your text editor and write your system prompt there you know let it be big because they can be big and then fa it into the tiny little text box when you're done yeah no I think that's excellent
1:37:02advice um and then so to your point Jeremy like I usually just put my transcript in here like a from Circle back um and then that's the transcript of your conversation with the client in which you're describing process yeah conversation with the client from Circle back and just one shot a proposal and you asked me about this like oh why do you one shot the proposal and it's because most people don't read The Proposal like it's a transactional thing you don't really sell work with
1:37:31proposals you sell work before you give them the proposal they've already decided whether they want to hire you or not proposal just a formality in a way and so this just removes that very tedious process which can take like an hour normally just do it with AI um not so maybe I'll share my screen and try and
1:37:59summarize this so here is a uh so I um Hamil and jono and I had a kind of trial run couple of days ago from which I generated this picture we can provide um of kind of Ham's process he described you know like trying to capture that authentic voice you know this is not so much for the Consulting project stuff but for the higher quality writing you know you can record anywhere you're
1:38:28trying to like get a brain dump in your authentic voice you know and then you can um you know use something like voice pal or an interviewer to kind of get follow up um ideally you have that transcribed at the same times using something like a super whisper on device or Circle back on Zoom otherwise transcribe it you know if you don't need the whole thing grab the bits you need
1:38:55um add add context so all the you know the the pairs of input and output transcriptions or samples of previous rating Etc at the first draft of the first little bit you know kind of iteratively looking at titles possible outlines Etc and then you kind of go into this hopefully positive feedback loop process um I'm terrible at creating these kinds of things so how did I get
1:39:21this well I got this uh from this text description of what Ham's been talking about and the text description very meta comes from using the solit process to analyze an interview with haml on the solver process and so here's a tool we haven't looked at that is really great which is Google AI
1:39:47studio um it's remarkably it's free for like more than any human could possibly use I reckon um you can add whole audio to it here a 50-minute interview with haml and so I started going through that process so I was like okay I I I can't remember everything haml said you know so go through all the tips and techniques again this is kind of the the diarization trick haml the guy who's
1:40:17answering the questions Jeremy's the guy who's asking the questions um now the first time I tried this this it did a terrible job of like and and I knew why it's because language models were until a few weeks ago bad at organizing things into groups because it requires more of a step-by-step approach so the trick was this was jonno's idea was to switch to Gemini 2.0 flash thinking experimental thinking is one of
1:40:45these ones like R1 and o1 O3 that goes through a reasoning process and as soon as I made that change um so you can see it's got this um thought process that it um documents here thinking process um and it actually did a really very good job of going through and so I uh the other nice thing about AI studio is you can edit the responses just I can solve it so you
1:41:15know much as I like solve it I would have to admit AI studio is also an excellent writing partner um solit actually doesn't support directly analyzing audio yet um so this is a really good thing to do so you can edit and then you can change it you know and then you can you know run a prompt and then that prompt will use the edited version for the in context learning so we edited a small
1:41:43amount um but again the trick here was not to say write something for me it's like write the list make sure you didn't miss anything um and then um what I then did was I uploaded that um markdown gist I showed you before of the conversation with Eric and said like hey this is something called Sol it you know tell me about how hamls approached writing and solve it a
1:42:11similar and again it did a really good job we might have deleted a little bit of stuff from here but we didn't edit it too much if I remember him yeah it's very nice they allow you to edit the output is it's great um so then with if they got that from your the idea from you or where originated from then uh um finally we're ready right and again using the kind of stuff Hamil's been
1:42:40talking about just WR the intro right and a bit of an overview and I think we didn't like this very much Hammer was like H this is not great so we started again like okay now instead come up with a title and so this come this I this uh prompt came from Ren actually which is just to grab titles and descriptions from our blog so in context learning um
1:43:09and I asked for 10 I always like asking for 10 and we looked through it and between you know H liked one best I liked one best we ended up kind of combining them and editing it slightly so we like okay start again redo everything you just did based on this total and this was much better don't you think H this version yeah it was a lot better yeah so it's like you saying the
1:43:38title so important um we edited a little bit and then again we're not like okay keep writing and stuff give me a plan for writing you know and and Hammer was particularly interested in like starting with mistakes people make I guess that's because like it's a useful idea in writing isn't it just to like show people why what this is about is that your thinking there H yeah yeah it's
1:44:06because there's likely that a lot of people are making those mistakes and that will Captivate you to read more oh yeah so anyway we're kind of going through this process and we ended up with this output and then I took that output and I passed the out put to R1 and I said to R1 generate a mermaid diagram and then I gave it some feedback and it iterated on the mermaid diagram um and it was interesting you
1:44:33know at some point then um jonno goes like um like what about this which is this like super concise version of it um and I like wow this is really good yeah I was impressed by this actually yeah
1:44:55and it was um you know it's just a one-shotted R1 thing which again I guess is partly the power of the reasoning model here well well partly also and and this is maybe my final objection that I didn't say is that I think the reason all one could do it frankly is cuz I gave it the bullet points right I said these are the things you should say exactly you it into like
1:45:24this beautiful thing and this is like this is my objection with your objections that happen like I swear weekly jono is you always say this like why don't you just do X and I was like because that's such a good idea that I could never have come up with like your your version that you gave R1 was marvelous it's like well okay step one be jonno be really good at just about
1:45:50everything and then so so I to try and defend um API the um the reason I the reason I could do that prompt is because we just spent like an hour talking about this thing right and I think the reason people enjoy writing and talk about writing as a tool for thought is because it forces you to spend a lot of time dwelling on the material and and thinking about the structure and so on
1:46:19and I think the key thing with the methods that we've shown today is that they don't shortcut that completely right because if you did you would never have that like I've thought about this enough that I know the right way to phrase it and sometimes like in the case of this thing I I at least felt like I thought about this now enough that I know the right way to phrase it and R1 or any model could just fine write the blog post and it's it's it's it's okay it's fine um but I think the trick
1:46:48is doing the stewing and doing the tackling and if you're doing that by a long voice note and then reading in the transcript and then having a discussion about it with the AI That's still engaging and I think that still helps you get to that point um it doesn't look as fancy as me saying oh excuse me I've done this amazingly um but it's that's like a lie right because you didn't just do it a second you did it in an hour and a second and my best writing um I had I had
1:47:16to um um I had to write really well when I was trying to convince the world to do something really weird which is to wear masks which white people don't do and um I basically learned to write well and I discover that the way you write well is by talking to really good writers go deep with the really good
1:47:44writers then have them critique every single word and every single sentence and every single paragraph of your draft and then do it eight times with really good writers and that's actually much better than using Claude or using Gemini to be honest you know and start with like having multiple multi-hour conversations with experts about the details so like you know yeah a lot of this and I think this is why often
1:48:10experts can quite poo poo AI process is about just about anything you know because they're it's not the same like it's not you know talking to Gemini was not the same as talking to zet you know like well she's a New York Times journalist you know she's the famous author of audle books and she's pretty good at this but yeah it's
1:48:37like it's still pretty good you know and you can talk to your friends who are not saying it too fetchy but they'll still be able to help still be able to VI to you you can still talk to an AI and get some assistance um so yeah you know it's a lot of the things we're describing I think about writing are absolutely not unique to AI they're certainly not our invention you know maybe calling it a solvid approach again
1:49:04it's like Eric was saying these are all versions of I don't know Japanese Manufacturing Systems or Lean Startup or the ud Loop in Airplane Warfare and and we're all still figuring them out right we're like kind of stumbling through and and sharing them because they're useful
1:49:25yeah okay well I think I'm out of things to say about writing we went a bit longer than usual even but I think it's an interesting topic I was worried we wouldn't have enough to say I guess we did oh wow and people really liked this everybody stayed on in the analytics so this is actually as good a retention as we had with the Eric discussions so people like thanks everybody thanks very much ham nice to have you as a special guest yeah my honor
1:49:53sorry that's the rules thank you Jon I hope we convinced you that maybe AI has some use in the world of writing even if I don't expect you to be using it in a hurry I guess so cheers everybody