Mid-range local AI build
This is the tier most people should aim for, and the one the rest of our build guides are measured against. A single 24 GB card handles mid-sized language models at useful context lengths, runs image generation comfortably including batches and upscaling, and gets you into short video clips and light fine-tuning. It is a single-GPU build with no unusual cooling or power requirements. Hardware covers working out what tier your existing machine already sits in.
Target
Roughly 16 to 24 GB of VRAM, or an Apple Silicon machine configured with 32 to 64 GB of unified memory.
Approximately, that gets you:
- Mid-sized language models, the tens-of-billions parameter class, at 4-bit quantization, with room left for a KV cache at a real working context.
- Small models at higher precision, if you would rather spend memory on quality than size.
- Image generation with batching, control models and upscaling, plus short clips from smaller video diffusion models.
- LoRA training on small and mid-sized models, which is where local fine-tuning becomes practical rather than theoretical.
These are approximations. A longer context or a less aggressive quantization can push a model that "fits" straight back out of memory.
GPU or unified memory
Three of the cards we cover land here:
- RTX 3090, 24 GB of GDDR6X. The memory capacity is the same as the tier above it, which is why it stays relevant long after its generation did.
- Radeon RX 7900 XTX, 24 GB of GDDR6. Same capacity class, different software story; check support for the apps you actually use before you commit.
- M3 Max in a mid unified memory configuration, where the GPU can address far more memory than a same-priced discrete card, at lower memory bandwidth than the fastest desktop parts.
Choose Apple Silicon here if you want one quiet machine that also does everything else, or if you care more about loading large models slowly than loading mid-sized ones fast. Choose a discrete card if you want speed, the broadest app support and a clear upgrade path.
The rest of the box
- CPU. A current midrange desktop CPU with eight or more cores. It matters for data preparation, tokenisation and inference on layers you offload, not for the main GPU path.
- System RAM. 32 GB minimum, 64 GB if you plan to hold multiple models or offload. Weights are staged through system memory on load, so thin RAM makes every model swap slow.
- Storage. A 2 TB NVMe drive is a sensible floor. Quantized GGUF files, diffusion checkpoints and their LoRAs accumulate quickly, and you want models on fast storage rather than a spinning archive drive.
- Power and cooling. Size the supply with clear headroom over the card's rated draw, and give the card room to breathe. Sustained generation runs the GPU hot for far longer than games do; a card that thermally throttles costs you throughput on every job.
Upgrade path
The enthusiast tier buys bandwidth more than capacity, plus the 32 GB step, which means faster generation and longer contexts on the same model classes. Stepping down to budget halves your memory and, with it, the model classes you can load at all.