Fine-tuning
Fine-tuning is continued training of an already trained model on your own data, so it adapts to a task, a format or a style. You start from published weights rather than from scratch, which is what makes it affordable outside a datacentre. See the glossary for the rest of the terms on this site.
The mechanics are the same as any training run. Examples go through the model, the output is compared against the target, and the error is pushed back through the weights. The difference from pretraining is scale: a much smaller dataset, far fewer passes, and a much gentler learning rate so the model does not forget what it already knew.
Why it matters for local AI
The memory cost is the thing that decides whether you can do it at home. Running a model for inference needs the weights plus a little working space. Training it needs the weights, a gradient for every trainable weight, and optimizer state on top of that. Full fine-tuning of anything but a small model will not fit in consumer VRAM.
That is why local fine-tuning almost always means parameter-efficient fine-tuning, and in practice LoRA. Only the adapter is trained, so gradients and optimizer state exist for a tiny fraction of the parameters and the job fits on a single card.
It is also worth knowing what fine-tuning is not for. It shapes behaviour: tone, output format, domain habits, refusal style. It is an expensive and unreliable way to insert facts, because there is no guarantee any particular fact survives the averaging.
Rule of thumb
- Exhaust prompting first, then retrieval with embeddings, then fine-tuning. Each step up costs more effort and more maintenance, and most problems stop before the last one.
- Data quality decides the outcome. A few hundred clean, consistent examples that all look like the output you want beat a large scrape of mixed-quality text.