Embedding

An embedding represents an item as a vector: an ordered list of numbers that a model can use. The item might be a word, a passage, an image, a user, or a product. In machine learning, embeddings are usually learned so that their relationships help with a particular task.

What the numbers represent

A vector with 384 dimensions has 384 coordinates. That is an example size, not a required range. Learned coordinates usually do not have individual names such as “refund” or “customer loyalty.” An embedding can be compact relative to a sparse encoding, but being an embedding does not require a universal dimension count or a reduction from every possible input representation.

The training objective determines which relationships become useful. Predicting words from their context, bringing matching image-text pairs together, and predicting purchases can produce different spaces. An existing model can generate embeddings for new inputs without being retrained for each one. Google's explanation emphasizes both task dependence and the difficulty of interpreting individual coordinates. Google: Embedding space

From a question to candidate passages

For document search, the preparation path embeds passages and stores the vectors alongside document identifiers. At request time, a compatible query encoder embeds the question, and a similarity measure ranks candidate passages. Query and document encoders may differ when trained to work together. Cosine similarity compares vector directions; a dot product also depends on their lengths unless the vectors are normalized. Sentence Transformers documents this query-and-document search pattern. Sentence Transformers: Semantic search

Suppose a customer asks, “Can I return opened headphones?” Imagine that a search system ranks a passage about returning unopened headphones above a passage about damaged products. That ranking may be useful for finding candidates, but it does not establish the answer. The first passage may explicitly exclude the customer's situation. These are illustrative search results, not measurements from a model.

The system must inspect the actual text, its effective date, and the user's access rights. A similarity score is not a calibrated probability that a passage answers the question. In a retrieval-augmented assistant, embeddings can help select context; they do not establish that the generated answer follows from it. Keyword search can also retrieve context, so RAG does not require embeddings.

Keep the representation and its sources aligned

Changing an embedding model can change the coordinate space. Equal vector lengths do not make vectors from unrelated models comparable. When replacing a model, rebuild affected stored vectors or establish and test a compatible migration path. Track the model, preprocessing, source version, and document-to-vector mapping.

Embeddings also support classifiers, recommendation candidates, clustering, and visualization. Their usefulness depends on evaluation for that use. Training data can encode unwanted associations, and proximity does not prove factual truth or fairness. A two-dimensional picture of an embedding collection is a projection, not a complete view of all relationships in the original space.