Guides to running AI models on your own hardware

A guide is a step-by-step how-to. It takes one question, such as whether a model will fit on your machine, and walks you from the start of the answer to the end of it. The reference pages elsewhere on this site describe what something is. A guide tells you what to do, in order.

That difference matters because the reference pages are organised by subject. A model page under /llms/models covers one model. A card page under /hardware/gpus covers one card. Real questions rarely stay inside one of those boxes: choosing a first model means reading hardware, model and runtime pages together. Guides do that joining up for you and link back out to the detail.

Start here

If you do not yet know what your machine can handle, begin at /hardware. It answers "what can I run on this?" and sorts machines by VRAM, memory bandwidth and whether the system uses unified memory. Almost every guide assumes you already have a rough figure for how much memory you can spend on a model.

Then go to /llms. A text model is the first thing most people run locally. It is the least fussy to set up, it needs no extra tooling beyond the app that runs it, and the ideas carry straight over to image, video and audio work. Pick an app from /llms/runtimes, pick a model from /llms/models, and you have something running in an evening.

Keep /glossary open alongside whatever you are reading. Terms like quantization, context window and GGUF turn up in nearly every guide, and each glossary entry is a short definition you can check in a few seconds rather than a page you have to read.

Planned guides

• Choosing your first local model. How to narrow the field down from thousands of options to two or three worth downloading.
• Will this model fit in memory?. Adding up weights, KV cache and overhead before you download anything.
• Picking a quantization. What you give up at each step down, and where the sensible stopping points are.
• GPU or unified memory machine. How the two approaches differ for inference, and which suits which workload.
• Running models on a laptop. Thermal limits, battery behaviour and keeping the rest of the system usable.
• Keeping everything offline. Checking that a local setup really does stay local, and what to watch for.
• Organising model files. A directory layout that survives multiple apps and stops you storing the same weights twice.
• Reading a model card. Making sense of parameters, licences, base versus instruct variants and fine-tuning lineage.

How guides relate to the rest of the site

The rest of the site covers text, image, video and audio, with /hardware and /glossary applying to all four. Guides are the horizontal layer. Each one links down into the subject, hardware and glossary pages it draws on, and those pages link back up here, so you can enter from any direction and still reach the answer in a couple of clicks. Nothing is explained twice: if a guide needs a definition or a spec, it points at the page that owns it.