Diffusion
Diffusion is a generative method that starts from random noise and removes it step by step until an image, video or audio clip emerges. Each step asks the model to predict what the noise looks like, subtracts a little of it, and feeds the result back in. See the glossary for the rest of the terms used across this site.
Training runs the process backwards. The model is shown real images with varying amounts of noise added and learns to predict the noise, which is the same thing as learning what the clean picture underneath should be. A text encoder conditions that prediction on your prompt, so the denoising walks toward images that match the description.
Why it matters for local AI
Diffusion demands different things from a GPU than text generation does. An LLM produces one token at a time and spends most of its time moving weights through memory bandwidth, which is why long generations feel bandwidth-bound. A diffusion model runs the same network repeatedly over a whole image at once, so it is compute-heavy and the wall clock scales with step count and resolution.
That changes the hardware picture. Diffusion models are usually smaller than large language models, so VRAM capacity is less often the wall. What you hit instead is raw compute, and resolution is the fastest way to hit it: the activation buffers grow with pixel area, and video generation multiplies that again by frame count.
It also explains the tuning knobs you see in local tools. Steps, sampler, scheduler and guidance strength all describe how the denoising path is walked, and they all trade quality against time.
Rule of thumb
- Treat steps and resolution as your two time dials. Halving the steps roughly halves the generation time; raising the resolution costs far more than the pixel count suggests.
- If a generation fails on memory, drop resolution before you drop the model. Tiled or staged decoding usually saves a run that a smaller model would only make worse.