Embedding Similarity

Embedding similarity is the step where a vector space becomes a ranking. You have embeddings for a query and for a set of candidates, and you need an ordering. The choice of measure is not a formatting detail: the same vectors produce different orderings under different measures, and the ordering is the product.

Three measures cover most systems. Cosine similarity compares direction, \(\cos(u,v) = \frac{u \cdot v}{\lVert u \rVert \lVert v \rVert}\), and lands in \([-1, 1]\). The dot product \(u \cdot v\) is the same numerator without the normalization, so increasing length raises a positive dot-product score but makes a negative one more negative. Euclidean distance \(\lVert u - v \rVert_2\) measures separation, and smaller is better.

Three measures, three answers

Four candidate vectors against one query, scored every way.

import numpy as np

rng = np.random.default_rng(0)
query = np.array([1.0, 1.0, 0.0])
items = {"same direction, short": np.array([0.2, 0.2, 0.0]),
         "same direction, long": np.array([5.0, 5.0, 0.0]),
         "slightly off, long": np.array([4.0, 6.0, 0.0]),
         "quite different": np.array([1.0, 0.0, 1.0])}

def cosine(a, b):
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

print(f"{'item':>24}  {'cosine':>8} {'dot product':>12} {'euclidean':>10}")
for name, v in items.items():
    print(f"{name:>24}  {cosine(query, v):>8.4f} {query @ v:>12.4f} {np.linalg.norm(query - v):>10.4f}")
print("\n  cosine ties the two same-direction items and separates the off-angle one;")
print("  the dot product does the opposite - it ties the two long ones and buries the short one;")
print("  euclidean ranks the short one first and the off-angle one last. Three orderings.")

print("\n  once every vector is unit length, cosine and euclidean are the same ranking")
print(f"{'item':>24}  {'cosine':>8} {'euclidean':>10}  {'2(1 - cosine)':>15} {'euclidean squared':>18}")
unit = lambda v: v / np.linalg.norm(v)
q = unit(query)
for name, v in items.items():
    u = unit(v)
    c = cosine(q, u)
    print(f"{name:>24}  {c:>8.4f} {np.linalg.norm(q - u):>10.4f}  {2 * (1 - c):>15.4f}"
          f" {np.linalg.norm(q - u) ** 2:>18.4f}")
#                     item    cosine  dot product  euclidean
#    same direction, short    1.0000       0.4000     1.1314
#     same direction, long    1.0000      10.0000     5.6569
#       slightly off, long    0.9806      10.0000     5.8310
#          quite different    0.5000       1.0000     1.4142
#
#   cosine ties the two same-direction items and separates the off-angle one;
#   the dot product does the opposite - it ties the two long ones and buries the short one;
#   euclidean ranks the short one first and the off-angle one last. Three orderings.
#
#   once every vector is unit length, cosine and euclidean are the same ranking
#                     item    cosine  euclidean    2(1 - cosine)  euclidean squared
#    same direction, short    1.0000     0.0000           0.0000             0.0000
#     same direction, long    1.0000     0.0000           0.0000             0.0000
#       slightly off, long    0.9806     0.1971           0.0388             0.0388
#          quite different    0.5000     1.0000           1.0000             1.0000

Every pair in that table disagrees with another. Cosine gives the short and long same-direction vectors an identical 1.0000 and separates the off-angle one at 0.9806. The dot product does the reverse: it ties the long same-direction vector and the off-angle one at exactly 10.0000, because length compensates for the angle, and it ranks the short vector last at 0.4000 — twenty-five times behind, on nothing but magnitude. Euclidean distance puts that same short vector first.

There is no measure-independent fact about which candidate is most similar. What the system means by similar is whatever the measure encodes, and length usually encodes something real: in many embedding models a longer vector corresponds to a more frequent or more confidently represented item. Whether you want that in your ranking is a product decision, taken once, and written down.

Normalizing collapses the choice

The second block shows what unit-normalizing does. Once every vector has length 1, the two same-direction candidates become the same point, and \(2(1 - \cos)\) equals the squared Euclidean distance exactly — 0.0388 and 0.0388, 1.0000 and 1.0000. This is an identity, not an approximation: expanding \(\lVert u - v \rVert^2 = \lVert u \rVert^2 + \lVert v \rVert^2 - 2\,u \cdot v\) with both norms equal to 1 gives \(2 - 2\cos(u,v)\).

For exact search on unit-normalized nonzero vectors, cosine similarity and dot product in descending order agree with Euclidean distance in ascending order, including ties. In an implementation, compare speed, supported score conventions, numerical behavior, and approximate-index recall as well. Sentence Transformers makes exactly that recommendation: if a model ends with a normalization step, use the dot metric rather than cosine, since “dot product on normalized embeddings is equivalent to cosine similarity” and cosine will normalize a second time for nothing. Its default across models is cosine, with dot product, negative Euclidean distance and negative Manhattan distance as the other options.

This also explains why vector databases usually ask you to declare a metric when you create an index rather than at query time. Approximate nearest-neighbor structures are built around a particular geometry, and a store built for inner product does not answer cosine queries correctly unless the vectors were normalized on the way in. Normalizing at write time is what makes the two agree.

What a similarity score does not tell you

A cosine of 0.87 is not a probability, and it is not comparable across models. Different encoders produce different score distributions, so a threshold tuned on one model has to be retuned on the next; vectors from two unrelated models are not comparable at all, whatever their dimensions. Within one model, scores are useful for ordering candidates and poor as absolute evidence, so a retrieval threshold has to be set against labeled pairs from your own data.

High dimensionality is a second caution, and it is the same one that limits nearest-neighbor search generally: under distributions with many similarly contributing coordinates, distances can concentrate as dimension grows, making nearest and farthest distances less distinct relative to their scale. Ambient dimension alone does not determine the effect: dependence, anisotropy, and effective dimension matter. A learned embedding may preserve useful neighborhoods, but training does not guarantee lower-dimensional structure or protection from distance concentration. Check the distance distribution and retrieval quality on the intended task.

The measure sits downstream of the encoder, so it cannot repair the encoder. If the embeddings were trained on an objective that treats your two items as interchangeable, no similarity function will separate them. Evaluate the pair — encoder and metric together — on labeled examples from the task you actually have.

References

• Sentence Transformers documentation, Semantic Textual Similarity. sbert.net
• Sentence Transformers documentation, Utility Functions (cos_sim, dot_score, euclidean_sim, manhattan_sim). sbert.net
• NumPy documentation, numpy.linalg.norm. numpy.org