Embedding
An embedding is a list of numbers that represents a piece of text, an image or audio, positioned so that similar things land close together. Comparing two embeddings gives you a measure of how related their sources are, without either one having to share a single word. See the glossary for the rest of the vocabulary.
Embeddings are produced by a model trained for exactly that job. Feed it a sentence, a paragraph or a document chunk and it returns a fixed-length vector. Two vectors that point in nearly the same direction mean two passages about nearly the same thing.
Why it matters for local AI
This is what makes local search and retrieval work. You chunk your notes, code or documents, embed each chunk once, and store the vectors. At query time you embed the question, find the nearest chunks, and paste them into the prompt. The model then answers from text it can actually see, which sidesteps the limits of a context window and avoids fine-tuning entirely.
The practical appeal is that embedding models are small. They run comfortably next to a chat model, often on CPU, and the indexing pass is a one-off cost. Nothing leaves your machine, which is usually the whole point of doing retrieval locally against personal or company documents.
The same idea shows up elsewhere. Image embeddings power local photo search by description, and audio embeddings sit behind speaker matching in voice cloning tools.
Rule of thumb
- Chunk on meaning, not on character count. Sections, functions and paragraphs retrieve better than fixed slices that cut sentences in half.
- Use one embedding model for the whole index and keep using it. Vectors from different models are not comparable, so changing the model means re-embedding everything.