Llama models for local use
Llama is Meta's family of open-weight language models, and it is the reference point for most of local AI. It is listed in the model index, part of local LLMs. Llama is known less for winning any single comparison than for being everywhere: new runtimes support it first, tooling assumes it, and the great majority of community fine-tunes and adapters are built on top of a Llama base. Weights are published under Meta's own community license rather than a standard open source licence, so read the terms before you build a product on it. For personal and local use it is permissive in practice.
Sizes and what they fit
Llama is released in three broad classes, and which one you pick is mostly a question of VRAM.
- Small, in the single-digit billions of parameters. Runs on almost any modern discrete GPU, on integrated graphics with patience, and comfortably on an Apple silicon laptop. This is the class to start with if you are unsure.
- Mid, in the tens of billions. Needs a card with a large frame buffer or a machine with a lot of unified memory. A 24 GB card such as the RTX 4090 handles this class at moderate quantization, and an M3 Max with a high memory configuration does too, more slowly but with room to spare.
- Large, in the high tens of billions and above. This is multi-GPU, workstation or heavily quantized territory. It will technically load on less, partly in system RAM, but the speed drop is severe enough that most people stay in the mid class instead.
Leave headroom beyond the weights themselves. The KV cache grows with how much context you use, and a long conversation can add gigabytes on top of a model that looked like it fit.
Runtimes
Every major local runtime loads Llama.
- llama.cpp was written for this family and still sets the pace for the format and the kernels.
- Ollama wraps it in a pull-and-run workflow and keeps Llama builds in its library.
- LM Studio gives you the same thing with a desktop UI and a model browser.
Quantized Llama weights are distributed as GGUF files, which all three read. Full-precision weights in the original format are mainly of interest if you plan to train.
Strengths and typical uses
Llama models are generalists. They hold a conversation, summarise, rewrite, answer questions about pasted text and follow instructions reliably enough for scripted work. Instruction-tuned variants are the ones you want for chat and assistant tasks; base variants are for people doing their own fine-tuning.
The family's real advantage locally is the ecosystem around it. If you want a domain-specific model, someone has probably already trained a LoRA adapter on a Llama base, and the inference stack you use will support loading it. That depth of tooling is why Llama is usually the safe default for a first local setup.