A large steel ceiling fan turning in a tall empty loft lit by high clerestory windows

RTX 5090 for local AI

The RTX 5090 is NVIDIA's Blackwell-generation consumer flagship and the largest single-GPU memory pool you can buy without moving to workstation or datacentre parts. It suits people who keep running out of room on a 24 GB card, particularly on video work and larger text models. Start from the GPU index if you are still choosing, and the hardware page for whole-machine planning.

Memory

The RTX 5090 ships with 32 GB of GDDR7 memory on a wider bus than the 4090. GDDR7 plus the wider bus means a meaningful step up in memory bandwidth as well as capacity, and for local AI both of those matter: capacity decides what loads, bandwidth decides how fast it generates.

The jump from 24 GB to 32 GB of VRAM is a third more room, which lands differently depending on the workload. All of the following are approximate and shift with quantization, context window length and KV cache growth:

  • Text models in the 30B parameters range move from tight to comfortable, with long context instead of rationed context.
  • The 70B class becomes genuinely usable at 4-bit rather than a heavily compressed experiment.
  • Image work gains headroom for higher resolutions, larger batches and more LoRA adapters at once.
  • Video is where the extra 8 GB is most visible, since diffusion video models scale memory with frame count and resolution.

What it runs well

  • Text (/llms): comfortable, and comfortable at a size class above what 24 GB handles well. Single-card fine-tuning also gets easier.
  • Image (/image): comfortable. Very little in current image tooling troubles 32 GB.
  • Video (/video): workable, and the best consumer option. Longer clips and higher resolutions than a 24 GB card, though video will still be the first thing to run out of memory.
  • Audio (/audio): comfortable. Audio models are small enough that the card is never the constraint.

Runtime support

The 5090 is a CUDA card, so the whole NVIDIA-first ecosystem applies, with one caveat: a new architecture needs runtimes and prebuilt wheels compiled against a CUDA toolkit that knows about it. Mainstream projects catch up quickly, but if you build from source or pin old dependencies, check that your toolchain targets Blackwell before assuming an inference problem is the model's fault.

Trade-offs

This is the most demanding consumer card to host. It draws more power than the 4090, so the power supply is a real line item rather than an afterthought, and it needs a modern 12V high-power connector run cleanly, fully seated and without tight bends at the plug. Sustained AI loads hold the card near its limit for far longer than gaming does, so case airflow and ambient temperature matter more than the spec sheet suggests. Partner designs are large; check length, slot count and front-radiator clearance before buying.

Against its siblings: the RTX 4090 remains the better value if your workloads already fit in 24 GB, and the gap is mostly headroom rather than capability (4090 vs 5090). The RTX 3090 is the opposite end of the same trade, buying capacity cheaply and paying in speed and efficiency. Two cheaper cards can beat one 5090 on total memory, at the cost of a harder build and split-model overhead; the 5090's advantage is that 32 GB is one pool, with no splitting and no interconnect to worry about.

Related