# Retrieval-Augmented Generation (RAG)

AI Glossary · Retrieval & RAG
Canonical URL: https://aisat.uk/glossary/retrieval-augmented-generation
Author: Lucy Gilmore, Assistant General Counsel, Product, Privacy & AI at Sprinklr | Practical Privacy, Product & AI Governance & Defensible Risk Decisions (https://www.linkedin.com/in/lucy-gilmore/)
Breadcrumb: [Glossary](https://aisat.uk/glossary) > [Retrieval & RAG](https://aisat.uk/glossary?category=rag) > Retrieval-Augmented Generation (RAG)

## TL;DR

> **Retrieval-Augmented Generation (RAG)** is an architecture that grounds a language model in external data before it generates an answer. By retrieving relevant passages from your documents and injecting them into the prompt at runtime, RAG cuts hallucinations, cites verifiable sources, and keeps answers current without retraining.

## How Retrieval-Augmented Generation (RAG) works in practice

A production RAG pipeline has two distinct stages: indexing and runtime retrieval. During indexing, source documents such as PDFs, Confluence pages, and SQL records are split into semantically coherent chunks, typically 300 to 800 tokens long. Each chunk is passed through an embedding model to produce a dense vector and stored in a vector database alongside metadata like document ID, access-control tags, and publication date.

When a user asks a question, the system converts the query into a vector and runs a hybrid search combining dense vector similarity with lexical BM25 matching, enforced against the user's permission filters. A cross-encoder reranker then scores the top twenty candidates and selects the five most relevant passages. These passages are injected into the model's context window with strict system instructions to answer solely from the supplied evidence and cite each source.

For example, a London commercial law firm uses RAG across 400,000 executed contracts and Practical Law updates. When a solicitor asks how force majeure clauses were negotiated across retail leases in 2025, the pipeline retrieves only clauses the solicitor's practice group is authorised to view, returning a cited comparison in four seconds.

## Why it matters and trade-offs

RAG solves the three hardest operational problems with off-the-shelf language models: knowledge cut-offs, unverifiable claims, and data access control. Compared with fine-tuning, which bakes static patterns into model weights at high compute cost and cannot easily revoke a deleted record, RAG updates the moment a document is re-indexed and enforces row-level security at query time. For most enterprise knowledge assistants, customer support bots, and policy search tools, RAG is the default starting architecture.

The trade-offs are retrieval latency, token cost, and pipeline fragility. Bad chunking splits tables or legal definitions in half, causing the retriever to miss critical context even when the answer exists in the corpus. Teams should not rely on naive vector-only RAG for multi-hop questions across hundreds of documents or whole-codebase reasoning, where GraphRAG, agentic search loops, or long-context models with prompt caching perform better.

## Related AI concepts

- [Vector Database](https://aisat.uk/glossary/vector-database): A Vector Database is a specialised storage engine, such as pgvector, Qdrant, Pinecone or Weaviate, built to index high-dimensional vector embeddings using approximate nearest-neighbour graphs and execute low-latency similarity queries alongside metadata filters.
- [Embeddings (Vector Embeddings)](https://aisat.uk/glossary/embeddings): Embeddings (Vector Embeddings) are dense arrays of floating-point numbers generated by an embedding model to represent the semantic meaning of text, code, images or audio so conceptually related items sit close together in vector space.
- [Hybrid Search](https://aisat.uk/glossary/hybrid-search): Hybrid Search is a retrieval pipeline that executes sparse lexical keyword search alongside dense semantic vector search and merges the candidate rankings using reciprocal rank fusion or a cross-encoder reranker.
- [GraphRAG](https://aisat.uk/glossary/graphrag): GraphRAG is an advanced retrieval-augmented generation architecture that combines vector similarity search with a structured entity-relationship knowledge graph to answer complex multi-hop and thematic questions spanning entire enterprise document collections.