Video generation on your own hardware

Generating video is generating many images that have to agree with each other. That single difference explains almost everything else on this page.

It explains why video needs far more memory than generating a still image does: the model holds a whole sequence at once rather than one frame. It explains why clips are measured in seconds. And it explains the failure you will actually hit, which is a drift rather than a crash, where a face is convincing in the first frame and has quietly become someone else by the last.

Expect to generate a lot and keep a little. That is normal here, and it is the reason running these models yourself makes sense: the cost of a bad attempt is a few minutes instead of a fee.

What the model is doing

A video model does not work on pixels directly. An encoder compresses frames into a much smaller representation, the model works in that compressed space, and a decoder expands the result back into something you can watch. Without that step the problem would not fit in consumer memory at all.

Inside that space the model is solving two problems at once. Each frame has to look right on its own, and the sequence has to look like something that happened. Those pull against each other, and when one loses you get the artefacts people complain about: flicker, morphing, a limb that changes length between frames.

Knowing that is practical, not academic. When a clip comes out wrong, the useful question is which of the two demands lost, because the fix is different for each.

Three ways in

The approach you choose matters more than the model you download.

Text to video starts from a description and nothing else. It is the hardest to control, because nothing fixes the first frame. You are specifying composition, subject and motion all through one paragraph of text, and the model resolves the ambiguity however it likes.

Image to video starts from a still you already have. Composition, colour and subject are settled before motion enters, so you are asking one question instead of four. For most people this is the faster route to something usable, and it is where I would start.

Frame interpolation fills in frames between frames you already have, to smooth or lengthen a clip. It generates no new content, which makes it far cheaper than another generation run. People tend to discover it last when it would have helped them first.

They combine well: make a still worth animating, animate it, then smooth the result. The techniques index covers each in depth.

The models

Five open-weight families cover most of what is done on home hardware, and they differ by temperament more than by quality. Some are built for speed, which lets you iterate. Some are built for fidelity and ask you to wait. One is not a standalone video model at all but a motion module that runs on top of an image model you already have, so it inherits the look you have already tuned.

The models index compares them: Stable Video Diffusion, AnimateDiff, LTX-Video, HunyuanVideo and Wan.

Which family is strongest changes faster than this page can track. The categories are stable, so learn those and treat any given model as the current best example of one.

What it asks of your hardware

VRAM decides what you can run. The thing that catches people here is that a model too large for your card does not simply run slowly, it fails to load, and there is no patience that fixes it. Check before you download with will this model fit in memory.

Once you are running, three things trade against each other: clip length, resolution, and how long you will wait. Push one and the others give. Most frustration with local video comes from expecting all three together.

Quantization is how larger models come within reach of ordinary cards. It costs some fidelity, usually less than people expect, and it is often the difference between a model you can run and one you can only read about. The GPU pages and the build guides cover what specific hardware can carry.

Where to start

Getting started covers the ground underneath this page. It assumes you have generated an image before and nothing beyond that.

Then take a still you already like, from an image model or a photograph of your own, and animate it. Keep the clip short. Expect several attempts to come out wrong in ways you did not predict, because that is the medium rather than a problem with your setup.

Most of this work happens in ComfyUI, where these models arrive as workflows you can open up and read.