How to Make AI Videos: The Full Workflow, Local GPU vs Cloud, and What It Really Costs

From our logs, October 15–19, 2026:

• 2,120 renders on one GPU: 1,383 images and 737 video clips (66 minutes of footage)
• €501.60 what the same output would have cost on pay-per-use APIs at mid-range list prices
• €2.06 of electricity for all of it (9.4 kWh)
• $48.52 total spent on paid APIs across every test, benchmark and paid shot

The short answer

To make AI videos that hold together, write the story first, lock one reference image per character, generate every shot from an image (not from text alone), cut in an editor, and put all text, product screens and claims in during the edit, never inside the generation. The model you pick matters less than that order.

On cost: a 32 GB consumer GPU makes images and short clips for the price of electricity, so you can afford hundreds of takes per shot. In our test week that was about 2 euro cents of power for every euro of cloud-equivalent output. Cloud models still win on acted drama: real facial performance, physical action and lip-synced dialogue in a single take. We used both.

• Local is best for: volume, iteration, B-roll, narrated and documentary-style videos, product scenes, motion graphics, music, and anything you'd otherwise pay for per take.
• Cloud is best for: the two or three hardest shots in a film: an actor's reaction, a crash, two people touching. At Kling 3.0 prices, those shots cost $1–2.50 each.
• The workflow beats the model. Our most "engineered" film met every technical spec and was still judged "terrible at any level of detail". A $4.70 plain-text test beat it on the scores that mattered.

The real project behind this guide

DipLeap turns your birth date, time and place into one calculated "life line" and asks you to check it against the years you've already lived before you trust it. It needed short-form video for Instagram and Facebook: emotional reels that bring reach, plus explainers that show the product. We decided to make them on our own hardware and to document everything for this guide.

Over two weeks we made five reel projects:

The rest of this guide is the workflow those projects converged on, with the numbers.

What you need: hardware and software

The GPU: why 32 GB matters

We run one NVIDIA RTX 5090 (32 GB VRAM) passed through to a Linux virtual machine with 64 GB of RAM. Under a three-minute burn test it sustained 215 TFLOPS in bf16 at 575 W, with no measurable penalty from virtualization.

If you're asking "what's the best GPU for AI video generation?", the honest answer from our logs is: VRAM decides which models you can run at all, but system RAM decides whether a long job finishes. The 32 GB card held every model we used (a 22-billion-parameter LTX video model in 8-bit, Wan 2.2 14B, Qwen-Image-Edit, Z-Image). What broke was RAM:

A few things mattered if you virtualize the card, as we did:

• UEFI firmware, not legacy BIOS. The 5090 has a 32 GB memory window (BAR) that legacy BIOS often can't map.
• NVIDIA's open kernel modules and a current driver. Blackwell cards need them.
• Test the reset. Some 2025 setups lost the card after a VM shutdown. Ours came back cleanly after both clean and hard stops, and we tested that before relying on it.

The software stack (all local, $0 per render)

Everything runs in ComfyUI, the node-based open-source tool most local video work uses, driven by Python scripts through its API. Here's what each model did for us and the licence to check before client work:

How to make AI videos: the workflow, step by step

This is the order we use now. It took two failed films to learn it.

Step 1: Write the story hook-first

Before any GPU time, write a timed beat sheet: what happens in each second. For short-form reels the structure that held up for us was:

1. Cold open on the peak moment (0–3 s). Not the setup, the crash, the reveal, the surprising line.
2. A question hook over it. "Could he have known?"
3. Rewind into the story that answers it.
4. A second hook at 10–20 seconds. Our Rembrandt reel lost its audience at about 10 s, right where a slow montage began.
5. Payoff, then the product, with the call to action in the last 3–5 seconds, spoken and on screen.

Then make an animatic: rough frames cut to scratch voice-over and temp music at final length. We rendered ours from clay previs in Remotion in about 40 seconds. The owner judges the story and the edit here, before anything expensive happens.

Step 2: Prompt the scene, not the pixels

Our biggest single lesson: a plain, well-written scene description to a strong video model beat our whole engineered pipeline. For local LTX-2.5, these rules took the share of takes that cut exactly where the prompt said from "loose" to 8 of 8, dialogue to 3 of 3 word-for-word, and a CCTV angle to 5 of 5:

1. One present-tense paragraph of prose. No labelled fields, no shot lists.
2. Action first, with motion verbs. Openings without motion give static clips.
3. Two or three shots per 8–10 s take, about 3 s each. The local distilled model tops out around 10–12 s.
4. At every cut, name the transition ("A hard cut transitions to…") and re-establish shot size, angle, who's in frame and the light.
5. Reuse one visual tag for a recurring person ("the man in the mustard-yellow rain jacket").
6. Show emotion as physical cues ("his jaw loosens into a small smile"), not mood words.
7. Give every sound a visible source, and end with a concrete audio sentence.
8. Dialogue in quotes, about 2.5 words per second, with the language and accent named.
9. No negatives, no quality tags, no narration. Distilled models run at CFG 1, where the negative prompt is ignored, and "no text" can invite text. Narration gets burned in as a fake caption.
10. Never name cameras, lenses or "footage". The model paints the camera into the frame.

Step 3: Lock your references: one hero frame per character

Text-to-video invents a new person and a new street for every shot. Our first all-text draft had a different man in almost every clip. What fixed continuity, cheapest first:

• One approved hero frame per character. Qwen-Image-Edit then puts that same face into every location (shop, street, couch, crash, payoff) and makes new moments and angles of each scene from that still.
• Identity LoRAs, when you need a face or prop to survive dozens of shots. We generated about 20 varied images of each subject from one approved reference and trained small LoRAs on our own GPU: people at rank 32 for 1,500 steps (about 30 minutes each), props at rank 16 for 1,000 steps (about 20 minutes). After that the same man, woman, vase and car appeared in every keyframe.
• Release layout control halfway. Holding a pose/depth ControlNet at full strength to the end gave dark, mottled frames. Releasing it at about 50–60% of the steps kept the blocking and gave clean photographs. A 5-minute A/B test on two frames saved an hour of re-renders.

Step 4: Image-to-video: generate takes, not shots

This is where "image to video AI" does its job. Each shot is its own image-to-video take from its still, and you generate several takes and pick. Locally, at about 25–80 seconds per take, that's cheap:

• Three seeds per shot. In our tests the seed changed the action far more than any strength setting did.
• Start physical events from the "about to happen" still. Our crash landed in 4 of 4 takes when the take started on a still of the car two metres away, and in 0 of 10 when we described it from text alone.
• Chain dialogue scenes. Start the next take on a frame 12 frames before the previous take ends. The join was invisible every time, and the overlap gives the editor room.
• Use keyframes inside a take only to time an event. Two or three keys at strength 0.5, at least 48 frames apart. Strength 1.0 caused slow-motion "hangs"; tighter spacing caused pop-ins.
• Change camera angle in the edit, never inside a take. Neither chaining nor keyframes could move the camera; they produced dissolves.

Step 5: Run an independent QC pass

Whoever wrote the prompts can't judge the results. Our own review called a batch of 50 keyframes "strong". A separate AI reviewer that saw only the frames, the continuity notes and the shot list, never the prompts, scored them 2 pass, 21 weak, 27 fail: six different versions of the same man, six different vases, five different cars. On the 40 motion clips the same kind of reviewer scored 6 pass, 15 weak, 19 fail.

We now check three things automatically on every take:

• Speech-to-text on every spoken line (faster-whisper), compared with the script.
• Face similarity against the hero frame, with any score under 0.5 flagged. A wrong face can score negative.
• One human pass on the cut, judged as a viewer.

The cheapest fixes come first: ignore problems after the part of the clip the edit uses; reverse a clip whose action ran backwards; fix the keyframe in place (remove fake text, delete a duplicate person); only then re-render with one action per prompt and two seeds. For more on using one model to grade another, see our LLM-as-judge guide.

Step 6: Upscale and interpolate (in a separate pass)

Wan 2.2 renders at 720p and 16 fps on our setup, so its shots go through frame interpolation (RIFE, 16→32 fps) and an AI video upscaler (Real-ESRGAN ×2, 720→1440, then down to 1080). LTX renders native 1080p at 30 fps for short shots and skipped this step. Two rules:

• Draft at base resolution, upscale only the winner. Four or five seeds at base resolution are cheaper than one upscaled loser.
• Run upscaling after the models are freed (see the RAM callout above).

Our finishing order is: upscale → light blur → grain → grade. Run colour filters in RGB. Our first draft came out magenta because a filter ran on YUV video.

Step 7: Edit, captions, music and sound: where the claims live

Nothing that carries meaning is generated. Captions, prices, signs, dates, the logo and the product itself are added in the edit, from real assets:

• The real product, full-width. Generated phone screens are always fake. We captured the real DipLeap app with a headless browser at phone resolution and composited it, or cut to it full-width with the character's line still playing over it (a J-cut). That J-cut was the moment the owner said the product finally sat inside the story.
• Never put text into any generation input. Captions inside a control video came back as garbled fake glyphs. A LoRA trigger word ("v4se") once printed itself on a price tag.
• Blur what the model invents. Generated number plates and real car badges appeared even when the prompt said "no logos". A blur box that tracks the car fixes it.
• Sound is half the film. Silence before an impact, music entering on a freeze, room tone under every cut. Our synthetic voices were the weakest element; letting the video model speak the lines inside the clip (LTX-2.5 locally, Kling in the cloud) worked far better.
• Music. ACE-Step generates a 30-second instrumental in about 2 seconds. Generate two or three alternates and let a human choose by ear: "cinematic pop" was judged silly, a "pulsing cinematic score" was liked.

Step 8: Export for the platform

Our Instagram Reels delivery spec:

The real cost: local GPU vs cloud APIs

What one week of local rendering produced

We built a meter into our production dashboard. It counts every image and video the GPU wrote, estimates what the same output would cost on a pay-per-use API, and measures electricity from the card's own power readings.

How the "cloud-equivalent" figure is calculated: €0.04 per image (a FLUX-pro-class image API), €0.08 per second of video up to 1280 px, and €0.15 per second above it. These are mid-range 2026 list prices, not a quote from any single vendor. Measured average draw while busy was about 460 W.

One overnight experiment, priced both ways

For our continuity lab we ran about 230 LTX-2.5 takes overnight, plus 30 image edits and an animated 3D layout, all locally. The lab folder now holds 326 takes from that night and the scenario runs after it: 2,394 seconds of video, averaging 41 seconds of GPU time per take (20–94 s).

Derived from our run logs × list prices on the day we checked them (2026-10-08 and 10-17). Not every take would be needed in a paid workflow, but you can't run this kind of experiment on a per-clip budget.

What we actually paid the cloud

Every paid call went through a ledger with a hard cap enforced in code. Across the whole project (benchmarks, keyframes, paid shots and tests), we spent $48.52:

Cloud price sheet (list prices we checked, October 2026)

So what does AI video generation cost per finished minute? At Kling prices, a 40-second film of about 10 shots costs roughly $10–15 in motion fees if you keep only the takes you need. Our full paid version of The Crossing, 86 seconds with 11 paid shots, cost $26.13 including the test.

When does a local GPU pay for itself?

Our meter gives us the inputs: €501.60 of cloud-equivalent output in five days (October 15–19, 2026), minus €2.06 of measured electricity, is about €100 of net cloud-equivalent output per day at our usage intensity. Divide the card's price by that and you get the break-even in days of the same kind of use.

RTX 5090 prices as of October 2026:

Aggregator averages for new cards sit higher, around $6,000+ (price history).

Assumptions, stated plainly:

• Heavy daily use, at the intensity of our production week (hundreds of images and clips a day). At half that pace, double the days.
• The card price only. The rest of the PC (CPU, 64 GB+ RAM, storage, power supply) is excluded.
• Electricity is included, from the meter's measured draw at €0.22/kWh.
• Dollar prices are divided at roughly €1 ≈ $1 to keep the arithmetic simple. Adjust for the exchange rate on the day you buy.
• "Cloud-equivalent" uses mid-range list prices, not one vendor's quote (see the meter section above).

Two honest caveats:

• At a hobby pace of a few clips a week, the cloud is cheaper. Pay per clip.
• The cloud-equivalent is only "saved" if you'd otherwise have bought those renders. For the takes that ended up in our reels, we'd have paid. We wouldn't have run the 200-take experiments at all, and we'd know much less.

When the cloud still wins

We ran the same three hard shots, a terrified face, a crash and one person cupping another's face, through our local model and three paid ones.

• Local LTX-2.3: near-frozen. The crash never happened.
• Veo 3.1: real acting and physics. The face reacts, the vase shatters, the face-cup becomes a hug.
• Kling 3.0: about as good as Veo for roughly a third of the price. Watch for gibberish number plates.
• LTX-2.5 Pro (cloud): better than local, weaker on action. Its open weights made it the model we then ran locally.

Then we tried the radical version: no keyframes, no references, no pipeline, just the cold open written as plain text and sent to Kling 3.0. Two 14-second clips cost $4.70. The version where the model chose its own shots produced the crash we'd failed to get in three days, and both looked more like real footage than anything our pipeline made. Read the full head-to-head in our AI video model comparison.

Where local caught up: LTX-2.5, running on our card for free, fixed the thing LTX-2.3 couldn't do, spoken dialogue, and with the prompt rules above it delivered shot structure, CCTV and night scenes reliably. What it still missed was the physical event from text alone. Starting the take from an "about to happen" still fixed that, as the overnight lab showed.

Case study: producing the DipLeap reels

We produced the DipLeap reels with this pipeline, and the honest story is more useful than a highlight reel.

Attempt 1: the engineered film. We built a 40-shot, two-minute ad with a 3D previs set, ControlNet keyframes, trained identity LoRAs, independent QC agents and a scripted edit. It met every technical target: the same cast in every shot, no fake text, a correct Instagram export at −14 LUFS. The owner judged it "terrible at any level of detail": too long at 1:57, the crash it opened on was never shown, too dark, mostly "stills that wobble", thin sound, and an unreadable phone. We had optimised what we could measure, and engagement wasn't on the list.

Attempt 2: paid motion. Nano Banana keyframes and Kling 3.0 motion, 86 seconds, $26.13 in total. Dialogue was spoken inside the clips (speech-to-text confirmed every line), and the real app appeared full-width under a J-cut. Still scored 2/10 on storytelling and 0/10 on the crash, CCTV and stakes. We'd softened the crash into a near-miss to avoid refusals, and cut the beat that showed what the vase was worth.

Attempt 3: the overnight local lab. About 230 LTX-2.5 takes, $0, testing five continuity methods on five edge cases, one parameter at a time. It produced the recipe in Step 4 and two reels: The Crossing v3 (47 s, a real crash, CCTV, stakes, the warning on the real app, the payoff) and "How Did You Know?" (28.5 s, a recognition hook built for sharing).

Attempt 4: real footage. The owner filmed three phone clips of a sunset surf spot. A small CPU tool we wrote turns a JSON spec into a 1080×1920 reel with a virtual camera (eased crop, zoom and rotate, never showing a border), speed ramps, transitions, text styles and two-pass loudness, at about one minute per 15-second reel. On the GPU we added a cosmic "your sky → your line" sequence (12 s at 1080×1920 in about 100 s) and generated music. Of 31 variants, the owner liked 11. The rules that came out: a real event plus a direct question beats a statement; never draw 2D graphics over real footage; any product visual must be GPU-generated or the real app.

What Instagram said. The accounts were brand new, so treat these as signals, not statistics. The Rembrandt documentary reel got 114 views on an empty account, with attention dropping at about 10 seconds where a slow montage began (the full story is on DipLeap's Rembrandt case page). One surf reel got 18 views and lost viewers at second 5, because small surfers in a wide shot aren't a hook. Each finding became a rule in Step 1.

Pitfalls and how to fix them

Where 3D layout (Blender) actually helps

We first used Blender as a still-photo stand: clay frames to force each keyframe's composition. It caught real errors before any GPU time was spent: an actor facing away from his wife in the emotional climax, heads out of frame, a camera behind a stack of boxes. But it didn't make the film better, because the models still had to invent all the motion from a frozen frame.

Blender earns its place in motion and geometry: one street, several cameras (walking, a CCTV pole, road level), so every angle matches across cuts. Rendering clay frames from two cameras at key moments, restyling them, and using them as video keyframes gave us a street shot and a CCTV shot of the same place. The weak point: an untextured set reads as a model railway, so dress it or restyle it harder.

Use control only where the story needs it: the crash and the CCTV shot get layout, dialogue and emotion get multi-shot prompts.

Should you build this, buy it, or hire it?

• You're testing whether video works for your brand: use a cloud model and plain prompts. Ten shots at Kling prices are $10–15.
• You publish weekly and iterate on hooks: a 32 GB GPU pays for itself quickly, and free takes let you test variants the way platforms reward.
• You need a repeatable engine: scripts, a GPU lock, a cost ledger, independent QC and a dashboard. That's software, and it's the kind of AI-enabled tool we build for clients.

NerdHeadz designs and builds production AI systems: custom AI development, AI agents that run multi-step workflows like this one, and automation that turns a manual process into a pipeline. If you want a content engine like the one behind the DipLeap reels, or want to know whether one makes sense for you, talk to us.

Related reading: Types of generative AI models · Why local AI ecosystems are winning · Open vs closed AI models · Hugging Face development · Our UG Video case study

Sources

• Our production logs, October 2026: GPU meter, paid-API ledger, independent QC reports, owner review scores (NerdHeadz content engine).
• Lightricks LTX documentation (docs.ltx.io) and the ComfyUI-LTXVideo workflows; LTX-2.5 prompting guidance as published via fal, deAPI and Runware.
• Wan-Video/Wan2.2 and the alibaba-pai Fun model cards; QwenLM/Qwen-Image and Hugging Face model cards; Comfy-Org workflow templates.
• RTX 5090 prices as of October 2026: gpupoet.com October 2026 price report, bestvaluegpu.com price history, WindowsForum: RTX 5090 hits €6,499 in France (2026-10-16).
• Google Gemini API documentation (Veo, Omni, pricing) and fal.ai model catalogue and pricing, read 2026-10-08 and 2026-10-17.
• Film craft: Walter Murch, In the Blink of an Eye (rule of six); StudioBinder (breakdowns, shot lists, the 180° rule); Pixar story reels; TikTok and Google ABCD creative guidance.
• DipLeap product claims from dipleap.com, read 2026-10-18 and 2026-10-19.

Frequently asked questions