Local video generation models
This is the index of open-weight video models covered here, part of the video generation pillar. If you have not generated a clip locally yet, start with getting started with local video generation and come back once you have a runtime working.
What makes a video model different
A video model has to solve a problem an image model never faces: consistency over time. Generate twenty frames independently and you get twenty pictures of roughly the same thing, flickering. A video model works on all the frames together, denoising a compressed representation that spans the whole clip, so motion and appearance are decided as one thing. Most of them are diffusion models underneath, extended with an autoencoder that compresses across time as well as space.
That design has a direct consequence for your hardware. Memory use is not just the size of the weights, it also scales with how many frames you ask for and how large they are. A model can load fine and still run out of VRAM partway through a longer generation, which catches people out constantly.
The shapes they come in
Video models differ along a few axes that matter more than raw quality scores:
- Text-to-video or image-to-video. Some invent the scene from a prompt, some animate a still you supply, and several do both. Which you want depends on how much control you need over composition.
- Standalone or built on an image model. Most are self-contained. One works as a motion module layered over image checkpoints you already have, which keeps downloads small and your existing styles available.
- Light or heavy. This is the big split. The lighter models run on mainstream cards and let you iterate. The larger ones ask for more memory than most consumer hardware has comfortably, and people fit them with quantization and by offloading parts of the model to system RAM.
How to pick
Work backwards from your hardware and your patience. Check what you have against the GPU pages and run the numbers with will this model fit in memory before downloading anything, because video weights are large and the download is the slow part of being disappointed.
Then be honest about how you want to work. If you need many attempts to land a shot, a fast model you can rerun is worth more than a slow model with a better ceiling. If you are producing one clip carefully, the reverse holds. Starting light and moving up costs you nothing but disk space.
The models
- Stable Video Diffusion — image-to-video, modest requirements, a good first model.
- AnimateDiff — a motion layer for existing image checkpoints, so it reuses what you already have.
- LTX-Video — built for speed and iteration rather than maximum fidelity.
- HunyuanVideo — a text-to-video model with an image-to-video path, published in more than one size.
- Wan — another large text-to-video and image-to-video family, published in more than one size.
Whichever you choose, the techniques pages cover the parts that are the same everywhere: prompting for motion, driving generation from a still, and stretching a short clip with frame interpolation.