How to make realistic AI videos with Claude: the full workflow start from Zero
Before anything
AI can generate anything now. Dragons, spaceships, people who never existed.
But when I scroll, I get bored of videos that feel too AI, unless the storytelling really pulls me in. I’m not saying dramatic scenes or fantasy don’t work. I think they work best when the fantasy looks realistic.
Why realistic matters
There’s some neuroscience behind this. Your brain is constantly predicting what it will see next. Look at any big fantasy show. The dragons are impossible, but the camera, the light and the way people move are real. Nothing breaks your brain’s prediction, so you stay in the story.
What this guide covers
So while experimenting with generative models, my goal switched from “make an AI video” to make one where you forget it’s AI.
I want to show you a workflow to start making AI videos from 0, and share all the lessons I learned from paying for generation, so you can skip the unnecessary spending.
I picked the easiest format to start with: a UGC video. UGC = user-generated content, the casual selfie-style clips where someone talks to the camera. Little movement, mostly talking.
Then I ended up trying a harder version too: a one-take video with several people in it. I’ll cover that in Step 5. If you only want to start with short UGC videos, you can skip that part. And at the end of this article I attached the skill that follow the exact aigc workflow.
Step 1: Research before you generate anything
I asked Claude to find viral videos using treg (opens in new tab), specifically videos with a group of fashion or marketing people, and break down why they worked. About 12 keyword searches, about 30 transcripts, plus the top comments.
Then I had treg’s AI judge score 10 hook options out of 20. The winner scored 15.9. and then remix the hook with my own case and wording。
Research is the cheapest step, and it decides everything after it.
Step 2: Lock the cast (character sheets)
Why you need a reference
Give the AI a picture to reference so it can keep the actors consistent in the video.
If you only describe someone in words, or upload a random picture with lots of faces, the result can be very confusing. Faces change between shots and things move around. It turns into chaos when you regenerate, because the model doesn’t know which face to use.
The character sheet
The fix is a character sheet: one image per person, with the face on the left and the outfit from the front and back on the right.
I also tried a version with the face and outfit together, but that confused the AI too. This turned out to be the best combination: a clean face picture, and a body-and-outfit picture without the face.
How I made them
For this step I used Gemini 3 Pro Image (Nano Banana) through treg.
I found it hard to describe a look just by talking to the AI. If you have the same problem, look for a few faces with the features you want your character to have, and give those pictures to the AI. It’s much easier to build your cast that way.
Don’t worry about the prompt. I’ll share the skill I used to make the full video, so you’ll have a skill that helps you build these cast sheets and everything else.
Set up a GitHub repo first
One thing before you create any picture: set up a GitHub repo for all your assets. It’s a very efficient way to manage your pictures and scripts with Claude. Once you lock something, it gets updated and versioned in the repo.
It also makes every asset easy to reference. Each file gets a permanent link, so you can send the same link to different AI models. Video models need a public link anyway, and a link saves Claude from carrying the whole file around (more on that in Step 6).
I also asked Claude to build a simple page in the repo that shows every locked asset in one place, in the exact order they’re sent to the model:
Step 3: Lock the set and the props
Besides the characters, we also want the background to stay consistent. And if there are important objects in the video, like a laptop, a sword or headphones, we want those to stay consistent too.
The trick is the same: create an asset for each one, so the AI can see how it’s supposed to look. Here I used treg and Nano Banana again to create the background (where it’s happening) and the important objects in the scene.
Same idea indoors. For the UGC version I made a loft office plate, plus a reverse angle of the same room, so the camera can turn around without the room changing:
Step 4: Cast the voices (~$0.05)
Tool: Gemini 3.8 Flash TTS via treg
Claude proposed a voice type, pitch and pace for each character, plus a test line. I asked for it as a spreadsheet so I could compare. 18 test clips cost about a cent.
Why the voice matters
The voice can turn out to be very important. When you look at a face, you already expect a certain sound. If you let the AI pick at random, the voice might not match what you imagined, or it can sound a bit fake.
To make it realistic, I used treg to pull a list of relevant voices, listened, and picked the one that matched each character. Then I attached the voice file as a reference when generating the final video with, for example, Seedance. The video model still generates the sound, but it uses your voice file as a reference, so you have an idea of how the character will sound. It’s less of a blind box.
Lock early, retake less
Our goal for keeping video generation cheap is to lock as much as possible early, before the actual generation, so we avoid retakes. In video generation, retakes are the most expensive part.
Fix pronunciation with a reference clip
I also noticed that when there’s a special word, like a brand name, the AI can pronounce it completely wrong. What I did was use treg to generate a voice clip with the right pronunciation and use it as a reference. That helped a lot.
Step 5: Plan the shots
Let Claude interview you
In this step, you discuss your video with Claude and ask it to make a shooting plan for you.
It’s really hard for us to write a perfect prompt for AI. I think the best way is to tell Claude what you want, ask it to interview you, and let Claude draft the final prompt that goes to the video model.
Think like the director
Describe your story in detail and use your imagination. Imagine you’re the cameraman, and the director. You control all the actors: how they move, what happens, what their faces and body language look like.
If you have no idea, ask Claude to give you hints or interview you. You can also use my skill, which I’ll share at the end. A good shooting plan needs a lot of knowledge: how the camera moves, which camera and lens you’re using, how the light works. If you don’t have that experience yet, the skill will help you start with a better plan.
Trick A: Generate keyframes first
Besides the detailed shooting plan, two more things helped a lot. The first is generating keyframes for each scene and each important moment.
That way you know roughly how the video will look: the lighting, the characters, and how they relate to each other in the frame. Even though they’re still images, they help you lock down a lot. If something is off, changing it at this step is way cheaper than changing it in a generated video.
Trick B: Build a 3D scene for multi-person shots
The second thing: I was trying to make a one-take video (一镜到底) with several characters, and things can go very wrong with that type of video. It’s very hard to control everyone and keep things consistent and natural.
What helped was asking Claude to build a small 3D scene as an artifact: camera distance, where everyone stands, and timing, all readable straight from it.
It helped me think through how everything should happen. It’s also a very good reference for the AI, because it’s pure data, not a blurry description. You can change and rearrange things easily, and it’s the cheapest preview you can get. It’s almost free.
Before the 3D model, I also tried a simpler version: drawing stand-in boxes for the people and furniture straight onto the real street plate, so the AI could see where everything goes.
Step 6: Generate the video
No video until every keyframe is approved. A still costs about $0.15. A video draft costs dollars.
There are two flows, depending on what you’re making.
Flow A: a simple UGC talking-head video
Best for one person talking to the camera.
One keyframe of your character in the location (Gemini 3 Pro Image): about $0.15 per still.
The voice track from Step 4 (TTS): less than a cent per line.
Lip-sync (HeyGen Avatar IV): the still plus the voice becomes a talking video. About $0.05 per second at 1080p.
Edit: cuts and captions, done locally, free.
Same plate, same light, two different characters. Because the room is locked, I can test which person delivers the hook better without rebuilding anything.
Estimate: a 30-second UGC video is about $1.50 for one pass, so roughly $2–4 with a retake or two.
One tip: Avatar IV holds the face really well, but the camera is frozen. Add a slight handheld shake in the edit so it feels filmed by a person.
Flow B: a short video clip with movement
Best for walking, camera moves, several people, or anything that isn’t just talking.
What I send to Seedance
Seedance 2.5 doesn’t just get a text prompt. Each request goes out with a pack of reference files:
The first keyframe: the still image the video starts from. It sets the light, the framing and where everyone stands.
Every character sheet: one file per person (face plus headless outfit), so nobody’s face or clothes drift.
The plate and props: the empty location and any object that has to stay the same.
The voice clips: the mp3s from Step 4, including the clips for hard-to-pronounce words like brand names, so the model says them right.
The shooting plan: camera, lens, movement, light and action, written with Claude in Step 5.
Send GitHub links, not attachments
Every one of those files is a link from my GitHub repo, not an attached file.
This was actually treg’s advice from the start, and there are a few good reasons for it:
The video models need a link. Image models often return pictures as a giant block of encoded text (base64), but video models like Seedance expect an image URL. A public GitHub link works everywhere.
Requests stay small. One high-res image as encoded text is a few MB. Five character sheets, a plate, props and voice clips in one request can hit size limits or time out. A link is one short line.
It saves Claude tokens. This is where the money is. The video model charges per second of video, so links don’t change that price. But if Claude has to pass a whole image around as text, that’s hundreds of thousands of tokens per file. A link is a few dozen.
One version of everything. If I fix a character sheet, I update it once in the repo, and every later request uses the new version.
The steps
Keyframes for every important moment: about $0.15 each, about $1–2 with retries.
Draft at 480p (Seedance 2.5, with the reference pack above): about $0.10 per second, so a 10-second draft is about $1.
Review the draft, fix the shooting plan or the asset, re-draft. Usually 2–3 rounds.
Final at 720p (Seedance 2.5): about $0.23 per second, so about $2.30 for 10 seconds. Only render the shots you approved.
Edit, locally and free.
Estimate: a 10–15 second clip is about $5–9, including drafts.
For quick, cheap motion tests, Hailuo 2.3 Fast is about $0.19 for 6 seconds at 768p.
Frames from my first 480p draft, a 14-second one-take with all five people. It cost $1.66. The faces, outfits and street held. What needed fixing was camera stuff (too close to one face, staying on one person too long), and those fixes go into the shooting plan, not the prompt.
Things I learned the expensive way
The first keyframe decides the video
The first keyframe is the most important file in the pack. Seedance uses it as the reference for how the whole video will look.
So before you generate, check it carefully: the lighting, the background, everyone’s face, and the space between people. If something is off there, it will be off in the video too.
Lock the voice reference before generating
Make sure you have the right voice clips ready before you start. That includes the clips for hard words like brand names. Use the skill to enforce this, so you don’t start a generation without them.
Don’t over-write the prompt
Write the shooting plan together with AI, but don’t go into too much detail yourself.
If you describe every single movement, the AI can focus only on the movements you mentioned and ignore everything else. People end up standing still, with weird facial expressions, and it looks off.
A moderate-length prompt works best. When it gets too detailed but still doesn’t cover everything, things go wrong.
Recap
Research first. Let AI search, you pick the references.
Lock the cast. A clean face picture plus a headless outfit picture, one file per person.
Lock the set and props. Keep every asset in a GitHub repo and send links, not files.
Lock the voice, including a reference clip for tricky words.
Plan the shots with Claude. Let it interview you, think like a director, make keyframes, and build a 3D scene for multi-person shots.
Generate cheap first. Keyframes before video, 480p drafts before finals.
Finish locally, for free.
The more you lock before generating, the fewer retakes you pay for.