Qwen models for local use
Qwen is Alibaba's family of open-weight language models, listed in the model index with the other local LLMs. What sets it apart is range. Qwen is released across more size classes than most families, from models small enough for a phone-class device up to weights that need server hardware, along with coding-specialised and multilingual variants. Much of the family is published under Apache 2.0, with some of the largest releases under Alibaba's own licence, so check the specific weights you download.
Sizes and what they fit
The unusually wide size ladder is the reason to care about this family locally: there is generally a Qwen model that matches whatever machine you have.
- Tiny to small, from well under a billion parameters up to the single-digit billions. The sub-billion models run on integrated graphics, old cards and phones. The small class fits any modern discrete GPU with room to spare.
- Mid, in the low tens of billions. A 24 GB card such as the RTX 4090 holds this class at moderate quantization, and so does an M3 Max with a large unified memory configuration.
- Large, in the high tens of billions and above, including mixture-of-experts designs. Multi-GPU or workstation territory. Check total VRAM across cards, not per card, and remember that the weights are only part of the budget once you start using a long context window.
Because the ladder is so dense, the usual advice applies more strongly here than elsewhere: pick the largest size class that fits with headroom rather than squeezing the next one in at aggressive quantization.
Runtimes
Qwen has first-class support across the local stack.
- llama.cpp supports the dense models and the sparse variants, and the coding models alongside them.
- Ollama keeps Qwen builds in its library across several size classes, which makes it easy to try two sizes back to back on the same prompt.
- LM Studio offers the same models through a desktop interface, with the quantization level shown before you download.
The distributed format is GGUF, as with the other families here.
Strengths and typical uses
Qwen's strongest suits are multilingual work and code. Chinese and English are both well covered, with meaningful support for a long list of other languages, which makes it a reasonable pick if your work is not in English. The coding-oriented variants handle completion, explanation and refactoring of real files, and the small ones are light enough to keep loaded in the background while you work.
The general-purpose instruction-tuned models do the ordinary assistant jobs: question answering, summarising, structured extraction. The small end of the family is particularly useful for embedded and always-on cases, where the goal is low-latency inference on modest hardware rather than the best possible answer. Qwen bases are also widely used for fine-tuning, and adapter support in the common training tools is good.