Local LLM models
This is the model index for local LLMs. It covers the open-weight families you can download and run on your own machine, without an API key and without sending your text anywhere.
A model page on this site answers four questions, in the same order every time:
- What is it good at. The kind of work the family handles well, described plainly, with no leaderboard claims.
- What sizes exist. The small, mid and large classes the family is usually offered in, and what each class asks of your machine.
- Which apps load it. Whether you can run it through llama.cpp, Ollama, LM Studio or all three. See the runtimes index for the full list.
- What hardware it fits. Which GPU class or Apple silicon machine holds the weights comfortably. The hardware pages start from the other direction, from the machine you already own.
The families
- Llama: Meta's family, the most widely ported and fine-tuned open-weight line, and usually the first one a new runtime supports.
- Mistral: compact, efficient models from Mistral AI, popular when you want strong output from a small download.
- Qwen: Alibaba's family, broad in size range and strong on multilingual and coding work.
Each page is written at the family level. Individual releases come and go, but the shape of a family, its size classes, its licensing style and the way it behaves locally, stays roughly stable.
How to read model sizes
Model names carry two numbers that matter, and they are easy to confuse.
The first is the parameter count, written as something like 7B or 70B. That is the size of the model itself, fixed by whoever trained it. More parameters generally means more capability and more memory.
The second comes from quantization, the step that compresses those parameters to fewer bits each. A model's parameter count never changes, but its file size does, and quantization is what makes a large model fit on a consumer card at all. Quantized weights for local use are usually distributed as GGUF files, a single-file format that llama.cpp and the tools built on it read directly.
A rough way to plan: take the parameter count in billions, multiply by the bytes per parameter your quantization level uses, and add headroom for context and overhead. That gives you the memory floor. If the number lands above what your card has, drop a size class or a quantization level rather than hoping it will page in gracefully.