A well-used repair bench in an attic room with hand tools and a magnifier lamp in low afternoon sun

RTX 3090 for local AI

The RTX 3090 is NVIDIA's Ampere-generation flagship, and it remains the most common way to get 24 GB of CUDA memory into a desktop without buying new silicon. It suits people building a first serious local AI machine on a used card, and people stacking two cards for capacity. See the GPU index for the alternatives and the hardware page for how to think about the machine as a whole.

Memory

The RTX 3090 carries 24 GB of GDDR6X memory on a wide bus. That is the same capacity as the RTX 4090, on an older generation of memory and a slower core, so the class of model you can load is broadly the same while the speed of loading and generating is lower.

Capacity is what decides whether a model runs at all. How far 24 GB of VRAM stretches depends on quantization, on the context window you configure, and on how much the KV cache grows as the conversation gets longer. Roughly:

  • Text models up to about 30B parameters at 4-bit fit with room for long context.
  • The 70B class needs heavy quantization and a short context, and is better handled by two cards.
  • Image models, including large diffusion backbones, fit comfortably alongside several LoRA adapters.

The 3090 also kept an NVLink connector, which the 4090 dropped. In supported setups two cards can be bridged, and even without NVLink a pair of 3090s is a well-trodden path to 48 GB of total capacity for larger models and for fine-tuning runs that will not fit on one card.

What it runs well

  • Text (/llms): comfortable. Speed is lower than current-generation cards, but capacity is what limits most people, and that is unchanged.
  • Image (/image): comfortable. Generation is slower per image, and everything that fits on a 4090 fits here.
  • Video (/video): tight. Video models push both memory and compute, so expect short clips, modest resolutions and patience.
  • Audio (/audio): comfortable. Speech, music and voice models rarely strain a 24 GB card.

Runtime support

Ampere is fully supported across the CUDA ecosystem. It is old enough to be well-tested everywhere and new enough to be a first-class target for today's engines, so you will not be chasing compatibility patches for ordinary inference work.

  • llama.cpp for GGUF models, manual layer offload and multi-GPU splitting.
  • Ollama for a low-friction pull-and-run setup.
  • LM Studio if you want a desktop GUI rather than a terminal.

A few newer numeric formats and kernels target later architectures only. When a project advertises a feature as requiring Ada or Blackwell, the 3090 will usually fall back to a slower generic path rather than fail outright.

Trade-offs

The 3090 runs hot and draws a lot of power. Plan for a high-wattage supply and strong case airflow, especially with two cards where the upper one gets fed the lower one's exhaust. The GDDR6X modules on the rear of many board designs are known to run warm, so used cards are worth checking for dried thermal pads and heavy mining wear. Most partner cards are triple-slot and long, which rules out small cases and makes two-card builds a motherboard spacing question as much as a budget one.

Against its siblings: the RTX 4090 gives the same capacity with far more speed and better efficiency (3090 vs 4090), and the RTX 5090 adds capacity on top of that. The 3090's case is capacity per unit of spend, which is why it stays popular.

Related