Embeddings (Vector Embeddings)

Computers cannot natively tell that 'solicitor', 'barrister', and 'legal counsel' share closely related meanings, because as ASCII strings they have almost no characters in common. An embedding model, typically an encoder-only transformer trained on contrastive pairs of related passages, maps each piece of text to a coordinate in a space of 384 to 3,072 dimensions.

During training, passages that answer the same question or appear in similar contexts are pulled closer together in that vector space, while unrelated passages are pushed apart. At runtime, measuring how related two texts are reduces to a fast geometric calculation, such as cosine similarity (the angle between two vectors) or inner product. Modern embedding models also support Matryoshka Representation Learning (MRL), which packs the most critical semantic signal into the first few hundred dimensions so engineers can truncate vectors (for example, from 1,536 down to 512 dimensions) to save RAM with minimal accuracy loss.

For instance, a Glasgow e-commerce retailer embeds both product catalogue descriptions and customer support tickets, enabling semantic product search, automatic duplicate-ticket clustering, and near-instant routing to specialist support queues.

Embeddings are the foundational building block behind semantic search, RAG retrieval, clustering, anomaly detection, and recommendation engines. Choosing the right embedding model often moves RAG retrieval recall far more than swapping the downstream language model or vector database.

The operational catch is that embeddings are model-specific and opaque: vectors generated by OpenAI's `text-embedding-3-large` cannot be compared with vectors from Cohere `embed-v4` or an open-weight `BGE` model. Switching embedding models in production requires re-embedding and re-indexing your entire document corpus. Furthermore, dense embeddings compress an entire paragraph into a single vector, which frequently loses exact product SKUs, error codes, and legal clause numbers unless paired with lexical BM25 in a hybrid search setup.