LoRA
LoRA, short for Low-Rank Adaptation, is a method for adapting a model by training a small set of extra weights alongside the frozen original, instead of changing the original weights. The result is a compact adapter file that gets loaded on top of a base model at inference time. See the glossary for the rest of the vocabulary around local models.
The trick is in the word low-rank. Rather than learning a full update for every weight matrix in the network, LoRA learns two thin matrices whose product approximates that update. That approximation is far smaller than the matrices it modifies, so the number of trainable parameters drops by orders of magnitude while the base model stays untouched on disk.
Why it matters for local AI
Full fine-tuning means holding the model weights, their gradients and the optimizer state in memory at once. That puts it out of reach on most consumer cards. LoRA only ever computes gradients for the adapter, so training fits in dramatically less VRAM and finishes in a fraction of the time.
It also changes how you store and ship adaptations. One base model on disk can serve many adapters: a writing style, a code dialect, a character or art style for image generation. Adapter files are small enough to swap between runs, and many apps let you apply more than one at a time or scale their strength.
The tradeoff is capacity. LoRA nudges a model's behaviour and style; it is a poor way to push large amounts of genuinely new knowledge into one.
Rule of thumb
- If you want the model to sound different or follow a format, reach for LoRA first. If you want it to know facts it has never seen, retrieval with embeddings usually beats any kind of training.
- Train the adapter against the same base model you will run it on. Adapters are not portable across different base models, and results degrade when the base is quantized very differently from the one used in training.