A loosely coiled network cable lying unplugged on a bare wooden floor beside an empty wall socket

Keeping everything offline

By the end of this guide you will be able to set up a local AI install that does its work with the network off, and you will know which parts of a typical setup reach out to the internet and when. It is for people who handle material they cannot send to a third party, and for anyone who simply wants to know where their data goes. It is one of the how-tos on the guides page.

What this assumes

The appeal of running models yourself is that inference happens on hardware you control. Your prompts, documents, images and audio never leave the machine, there is no account to sign into, no usage to log, and no change of terms that can reach back and affect work you have already done. That property is not automatic though. It is a consequence of how you install and configure things.

Worth having first:

  • A machine that can hold the models you need. The hardware section and will this model fit in memory? cover how to tell.
  • An app to run models in. LLM runtimes, image runtimes and audio models cover the options, depending on the kind of model you care about.
  • An understanding of model files as ordinary files on disk. A weights file in a format such as GGUF is self contained, and once it is downloaded it needs nothing from the network. Choosing the quantization you want before you go offline matters, because swapping later means going back online.

What reaches out, in general terms

Local AI software is still software, and software on a connected machine tends to talk. In a typical setup the network is used for a handful of predictable things:

  • Fetching models, which is a deliberate download you initiate and which dominates the traffic by volume.
  • Update and version checks, often run automatically at startup so the tool can tell you a newer build exists.
  • Optional usage or crash telemetry, which varies by project and is sometimes on by default.
  • Web user interfaces that load fonts, icons or scripts from an external source rather than bundling them.
  • Extensions, nodes, plugins and model catalogues that fetch their own content on demand.

None of this involves sending your prompts to a model provider, because there is no provider. It is still traffic leaving your machine, so if your requirement is that nothing leaves, treat each item as something to check rather than assume.

Steps

  1. Decide what offline means for you. A machine that never touches the network is a different project from one that is simply not sending your work anywhere. The first needs planning, the second mostly needs a settings pass.
  2. Do your fetching in one deliberate session while connected. Download the app, every model you expect to want, and any extensions or supporting files, then confirm they all load before you cut the connection.
  3. Read the settings for update checks, telemetry and anonymous usage reporting, and turn off whatever you do not want. Do this per tool, since each project makes its own choices and defaults change between builds.
  4. Disconnect and run a full task end to end. This is the only honest test. Anything that was quietly depending on the network will fail now rather than later, while you can still reconnect and fix it.
  5. If you need a hard guarantee, enforce it outside the application. A firewall rule, a separate network profile or a machine with no network configured at all removes the need to trust any individual setting.
  6. Write down what you installed and which model files you have. On an air-gapped machine there is no convenient way to look something up, so a plain text inventory in the same place as your models is worth the two minutes. See organising model files.
  7. Plan how updates will happen. Schedule a connected session now and then, refresh what needs refreshing, verify the setup still works offline, and go back to disconnected.

Common mistakes

  • Assuming that because the model runs locally nothing can be sent anywhere. The model is local, the application around it is ordinary networked software.
  • Cutting the network before testing, then discovering a missing file with no way to fetch it.
  • Downloading one quantization and finding it does not suit the machine once you are already offline.
  • Turning off telemetry in one tool and assuming the setting applies to every other tool you installed.
  • Relying on application settings alone when the requirement is strict. If it must not talk, block it at the network layer.

Related