Budget local AI build
This is the entry tier in our build guides: a machine that runs small models on your own hardware without turning into a project in itself. It suits people who want a chat assistant that works offline, who generate images as a hobby, or who are testing whether local AI fits their workflow before committing more. Expect small language models and single-image generation to feel fine, and expect longer documents, video work and large models to be out of reach. Hardware for local AI is the better starting point if you are trying to work out what your current machine already handles.
Target
Roughly 8 to 12 GB of VRAM, or a base Apple Silicon machine with around 16 GB of unified memory.
At that memory class you can approximately expect:
- Small language models, the single-digit-billion parameter class, at 4-bit quantization.
- Modest context windows. The KV cache grows with context, so long chats eat the headroom your weights left behind.
- Image generation from common open diffusion checkpoints, one image at a time.
- Speech to text and small text to speech models, which are far less memory hungry than image or language work.
Treat all of this as approximate. Quantization format, context length and app settings move the line more than the headline model size does.
GPU or unified memory
None of the cards covered in GPU pages sits in this class. They all start at 24 GB, which is the next tier up. For this build you are looking at a current-generation midrange consumer card with 8 to 12 GB, and the honest advice is to buy the most memory you can rather than the fastest chip you can. A slower card with more memory runs models a faster card simply cannot load.
An Apple Silicon laptop is a reasonable alternative shape here, because unified memory is shared with the GPU rather than fixed at manufacture. A base configuration gives you a usable amount of model memory in a machine that is silent and portable, at the cost of slower inference than a discrete card. If you want the Apple route with real headroom, see M3 Max and the mid-range tier.
The rest of the box
- CPU. Any modern six-core or better part is enough. The GPU does the work. A weak CPU only shows up when you offload layers to system memory.
- System RAM. Aim for at least twice your VRAM. Models are read into system memory before they reach the GPU, and CPU offload, which spills layers that do not fit into RAM, is how you run something slightly too large at reduced speed.
- Storage. An NVMe SSD with real capacity. Model files run from a few gigabytes to tens of gigabytes each, and you will collect more of them than you planned. Load times track drive speed directly.
- Power and cooling. A quality supply with headroom above the card's rating, and a case with front-to-back airflow. Local AI loads the GPU steadily for minutes at a time, which is harder on thermals than gaming bursts.
Upgrade path
Moving to the mid-range tier roughly doubles your memory, which is the jump from small models to mid-sized ones, from short contexts to working ones, and from single images to batches. Nothing else in the box needs replacing if you sized the supply and the RAM sensibly.