AnimateDiff
AnimateDiff is not a standalone video model. It is a motion module that plugs into an image model you are already running, teaching it to produce a coherent run of frames instead of one picture. It is one of the entries in the video models index, part of the video pillar.
What it does
A normal image checkpoint has no idea that frames come in sequences. Generate the same prompt twice and you get two unrelated pictures. AnimateDiff adds a layer that looks across frames, so the batch is denoised as a sequence and the frames end up as versions of one moving scene rather than a pile of near misses.
The important consequence is that the picture is still being made by your image model. AnimateDiff supplies motion; the checkpoint supplies everything else. Swap in a different checkpoint and the clip looks completely different, with the same module doing the same job underneath.
That is why it fits so well over an existing pipeline. Your LoRA still applies. Your prompt style still applies. Your conditioning and control tooling still apply, because the graph around the module is the image graph you have already built. What you get is that pipeline, in motion.
It also means AnimateDiff inherits your image model's weaknesses. If the checkpoint is bad at hands or struggles with a subject, the clip will be bad at it too, several times over and now moving.
What it needs
The module itself is small next to the image model it rides on, so the headline cost is not the download. The cost is that you are now generating many frames in one pass, and all of them are resident during inference.
In general terms:
- VRAM use rises with the number of frames in the batch, so shorter clips are the first lever when memory is tight.
- The frame size is the second lever, and it behaves much as it does for still image generation.
- Time per clip scales roughly with frame count, which makes it well suited to short, repeated experiments.
It is easy on a mid-range card, largely because the underlying image model is one you were already fitting into memory., largely because the underlying image model is one you were already fitting into memory. Our GPU pages cover what specific cards hold.
Where it fits
Reach for AnimateDiff when you already have an image pipeline you like and you want it to move. It is the one that keeps your image checkpoint in the loop, so the clip carries the style you have tuned rather than the look baked into a dedicated video model.
Choose Stable Video Diffusion when you have one specific still and you want that exact frame animated. Choose LTX-Video when the priority is turnaround time, or when you want a dedicated video model's grasp of motion rather than a motion layer over a still-image model.
Because prompting drives the clip, text-to-video is the technique page that applies most directly here.
Running it
AnimateDiff is almost always run through ComfyUI, via a set of community nodes that insert the motion module into an otherwise ordinary image workflow. In practice you take a graph that already works for stills, add the module, set how many frames you want, and send the batch out to a video encoder node instead of to a single image save.
That continuity is the practical appeal: there is no separate video application to learn, and everything you know about steering your checkpoint carries straight over.