Frame interpolation
Frame interpolation invents the frames that sit between the frames you already have. It is the odd one out on the techniques hub: it does not generate content, it only generates motion between content that exists, which makes it the cheapest fix available in a local video workflow and usually the first one to reach for.
How it works
The model is shown two consecutive frames and asked what the moment between them looked like. It estimates how each part of the picture moved from the first frame to the second, then warps both frames towards the midpoint and blends them, repairing the places where the warp leaves holes. Run once, you get one new frame between every pair. Run again on the result and you double the count again.
Two things follow from that. First, the output is bounded by the input: interpolation can only redistribute what is already on screen, so it cannot introduce anything that was not in either frame. Second, it works on any clip, whatever produced it. A generated clip, a screen recording, an old video file: the process does not care, because the inference step only sees pairs of pictures.
The result can be used in two ways. Play the extra frames at a higher rate and the clip is the same length but smoother. Play everything at the original rate and the clip is longer and slower, which is the cheap route to a slow-motion look.
What it is good at
- Smoothing generated clips, which often come out with visible stepping between frames.
- Stretching a short generation into something long enough to use, without a second video run.
- Slow motion from footage that was never shot for it.
- Rescuing a clip whose content is right but whose motion judders.
- Running on modest hardware, including machines that cannot generate video at all.
Where it struggles
- Large motion. If something moves a long way between two frames, the model cannot tell which part went where. Fast pans and quick gestures produce smearing or a visible jump.
- Occlusion. When an object passes in front of another, the area behind it was never photographed. Interpolation guesses, and the guess is usually a soft blur that pulses once per real frame.
- Ghosting. Thin, high-contrast details such as railings, wires and text leave doubled copies of themselves in the invented frames.
- Garbage in. If the source clip already morphs or drifts, interpolation makes the morph smoother, not smaller. It will happily produce a perfectly fluid version of a hand turning into a different hand.
- Overdoing it. Push the frame rate far above the source and the result starts to look synthetic, with motion that is too even to read as real.
None of these are fixable at the interpolation step. If the result is wrong here, the fix belongs upstream in text to video or image to video, or in choosing a different model from video models.
Cost
This is by far the cheapest of the three approaches. The models are small, they only hold a couple of frames at a time rather than a whole block, and they need a fraction of the VRAM a generative video run does. Many will run on a CPU if you are patient, and on any card that handles still-image work they are close to an afterthought. That economics is the whole argument for the standard workflow: generate the shortest clip that contains the motion you want, then spend the cheap step making it smooth.