From text to vectors
An embedding model maps a piece of text to a vector of numbers. Texts with similar meaning tend to map to similar directions. The model is learned from data; the geometry used to compare two vectors is simple.
The formula
The numerator is the dot product. The denominator divides by the lengths of the two vectors, so only their direction matters. The value lies between −1 and 1, and 1 means the same direction.
A worked example
Use invented two-dimensional vectors: q = (3, 1), d1 = (6, 2) and d2 = (1, 4). For d1 the dot product is 3·6 + 1·2 = 20, and the lengths are √10 and √40, so s = 20 / (√10 · √40) = 1.000. For d2 the dot product is 3·1 + 1·4 = 7 and the lengths are √10 and √17, so s = 7 / (√10 · √17) ≈ 0.537. The document d1 is twice as long as q but points in the same direction, so it is judged identical; d2 points elsewhere.
Why direction and not distance
Vector length often reflects how long a text is, not what it means. Comparing directions removes that effect.
What it does not do
Similarity is resemblance, not truth. A retrieval step built on it generates candidates; relevance still needs constraints (a property must be active, a document must be permitted) and evaluation.
Background reading
Works by other authors that treat the underlying methods. They are not Menlo Tech publications.
- Manning, C., Raghavan, P. & Schütze, H. Introduction to Information Retrieval. Cambridge University Press, 2008.
Related
Case studies