Glossary

Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation is a pattern that grounds a language model's output in documents fetched at query time from an external knowledge store, rather than relying only on the model's parametric memory. The retrieved passages are placed in the prompt as context, so answers can cite current, private, or domain-specific data the model was never trained on. RAG reduces hallucination and lets you update knowledge by changing the index instead of retraining the model.

Embedding

An embedding is a fixed-length vector of floating-point numbers that represents the meaning of a piece of text (or other data) in a continuous space, where semantically similar inputs map to nearby vectors. Retrieval systems embed both documents and queries with the same model so similarity can be measured by distance. Common dimensionalities range from 384 (all-MiniLM-L6-v2) to 1536 or 3072 (OpenAI text-embedding-3).

Vector Database

A vector database stores embeddings together with their source metadata and supports approximate nearest-neighbour (ANN) search over high-dimensional vectors. It powers the retrieval step of RAG by returning the chunks whose embeddings are closest to the query embedding. Examples include pgvector, Pinecone, Weaviate, Qdrant, Milvus, and Redis with RediSearch.

Chunk

A chunk is a self-contained passage of source text, produced by splitting a document so each piece fits an embedding model's context and represents a single coherent idea. Chunk size and overlap are core tuning knobs: too large dilutes relevance, too small loses context. Each chunk is embedded and stored as one retrievable unit.

Chunking

Chunking is the process of segmenting documents into retrievable passages before embedding. Strategies range from fixed-size token windows with overlap to structure-aware splitting on headings, sentences, or semantic boundaries. Good chunking keeps related content together and avoids cutting mid-thought, which directly improves retrieval quality.

Semantic search retrieves documents by meaning rather than exact keyword match, using vector similarity between a query embedding and document embeddings. It finds relevant results even when the wording differs from the query, unlike lexical search. It is the dense-retrieval half of most hybrid RAG systems.

Lexical search (also keyword or sparse search) ranks documents by exact and weighted term overlap with the query, using algorithms such as BM25 or TF-IDF. It excels at exact identifiers, codes, and rare terms that embeddings may blur. RAG systems often combine it with semantic search to get both precise and meaning-based matches.

Hybrid search fuses lexical (BM25-style) and semantic (vector) retrieval, then merges the two ranked lists into one. Reciprocal Rank Fusion (RRF) is a common, score-free way to combine them. Hybrid search typically outperforms either method alone because each compensates for the other's blind spots.

Reciprocal Rank Fusion (RRF)

Reciprocal Rank Fusion is a method for combining multiple ranked result lists into a single ranking using only each document's rank position, scored as the sum of 1/(k + rank) across lists. Because it ignores raw scores, it merges lexical and vector results that live on different scales without normalisation. A typical constant k is 60.

Approximate Nearest Neighbour (ANN)

Approximate Nearest Neighbour search finds vectors close to a query vector without scanning every vector, trading a small amount of recall for a large speed-up. Index structures such as HNSW and IVF make sub-linear search over millions of vectors practical. The accuracy/latency trade-off is tuned through index parameters like ef_search or nprobe.

HNSW

Hierarchical Navigable Small World is a graph-based ANN index that organises vectors into layered proximity graphs and greedily navigates from coarse to fine layers to find nearest neighbours. It offers high recall at low latency and is the default index in pgvector, Qdrant, Weaviate, and RediSearch. Build cost and memory are higher than IVF, and key parameters are m, ef_construction, and ef_search.

Cosine Similarity

Cosine similarity measures the angle between two vectors, ignoring their magnitude, with 1.0 meaning identical direction and 0 meaning orthogonal. It is the most common similarity metric for text embeddings because relative direction captures meaning better than raw distance. Many vector stores expose it as cosine distance, defined as 1 minus cosine similarity.

Re-ranking

Re-ranking is a second-stage scoring step that reorders an initial candidate set with a more expensive, more accurate model before passing the top results to the generator. A cross-encoder re-ranker reads the query and each passage together, capturing interactions a single embedding cannot. Re-ranking sharply improves precision at the top of the list at the cost of extra latency.

Cross-Encoder

A cross-encoder is a model that takes a query and a candidate passage jointly as one input and outputs a single relevance score, rather than embedding them independently. Because it attends across both texts, it is far more accurate than the bi-encoder used for first-stage retrieval, but too slow to score a whole corpus. It is therefore used to re-rank a small candidate set.

Bi-Encoder

A bi-encoder embeds the query and each document separately into the same vector space, so document vectors can be precomputed and indexed for fast nearest-neighbour search at query time. It scales to millions of documents but is less precise than a cross-encoder because the two texts never interact during encoding. It is the standard first-stage retriever in RAG.

Top-k

Top-k is the number of highest-scoring chunks a retriever returns for a query, which then form the context window for the generator. Larger k raises recall but adds noise, cost, and the risk of exceeding the context budget; smaller k is precise but may miss supporting evidence. It is one of the most impactful and cheapest knobs to tune.

Context Window

The context window is the maximum number of tokens a language model can attend to in a single request, spanning the system prompt, retrieved context, conversation history, and the model's own output. RAG must fit its retrieved passages inside this budget alongside instructions. Exceeding it forces truncation or summarisation and can silently drop evidence.

Grounding

Grounding is the practice of constraining a model's answer to information present in the supplied context, so claims are traceable to retrieved sources. A grounded prompt instructs the model to answer only from context and to say so when the context is insufficient. Grounding is what makes RAG citable and is measured by faithfulness.

Hallucination

A hallucination is model output that is fluent and confident but factually wrong or unsupported by any source. In RAG, hallucinations arise when the model ignores or contradicts the retrieved context, or when no relevant context was retrieved. Mitigations include strict grounding prompts, faithfulness evaluation, and an explicit "I don't know" path.

Faithfulness

Faithfulness measures how well a generated answer is supported by the retrieved context, independent of whether the answer is also correct. A faithful answer makes no claim that the context does not entail. It is a core RAG metric (for example in RAGAS) and is the primary defence against hallucination in grounded systems.

Answer Relevancy

Answer relevancy measures how well a generated answer actually addresses the user's question, penalising responses that are incomplete, evasive, or padded with irrelevant detail. It is scored independently of faithfulness, because an answer can be faithful to context yet fail to answer the question. RAGAS estimates it by generating questions from the answer and comparing them to the original.

Context Precision

Context precision measures how much of the retrieved context is actually relevant to the question, rewarding retrievers that rank useful passages above noise. Low precision means the generator must sift signal from clutter, raising cost and hallucination risk. It is a retrieval-quality metric that complements context recall.

Context Recall

Context recall measures whether the retrieved context contains all the information needed to produce the reference answer. Low recall means the right passages were never retrieved, so even a perfect generator cannot answer correctly. It isolates retrieval failures from generation failures.

RAGAS

RAGAS (Retrieval-Augmented Generation Assessment) is an open-source framework for evaluating RAG pipelines with metrics such as faithfulness, answer relevancy, context precision, and context recall, often computed with an LLM as judge. It works from a dataset of questions, answers, contexts, and optional ground truths. It is widely used to gate RAG quality in CI.

LLM-as-Judge

LLM-as-judge is an evaluation technique that uses a language model to score or compare outputs against criteria such as faithfulness, relevance, or correctness, replacing slow or unavailable human grading. It scales evaluation but introduces its own biases (position, verbosity, self-preference), so judges should be calibrated and ideally differ from the model under test. It underpins most automated RAG metrics.

Prompt Template

A prompt template is a parameterised text structure that combines fixed instructions with slots for the user query and retrieved context, producing the final prompt sent to the model. Keeping it in one place makes grounding rules, formatting, and few-shot examples consistent and testable. Templating libraries and prompt-management tools version these alongside code.

System Prompt

A system prompt is the top-level instruction that sets a model's role, scope, output format, and guardrails for an entire conversation, separate from individual user turns. In RAG it typically enforces grounding ("answer only from context") and citation behaviour. It is the main lever for controlling assistant behaviour without fine-tuning.

Retriever

A retriever is the component that, given a query, returns the most relevant chunks from the knowledge store, usually by embedding the query and running ANN search, optionally with hybrid scoring and re-ranking. Its quality sets the ceiling for the whole pipeline: the generator cannot use evidence the retriever never surfaced. It is the most common place RAG quality is won or lost.

Generator

The generator is the language model that produces the final answer from the user query plus the retrieved context and prompt instructions. Its job is to synthesise and cite, not to recall facts from training. Generator behaviour is shaped by the system prompt, the context quality, and decoding settings like temperature.

Indexing Pipeline

The indexing (ingestion) pipeline is the offline process that loads source documents, cleans and chunks them, computes embeddings, and writes chunks plus vectors and metadata into the store. It runs ahead of query time and is re-run when sources change. Idempotent, incremental indexing keeps the store fresh without full rebuilds.

Query Expansion

Query expansion rewrites or augments the user's query to improve retrieval, for example by adding synonyms, generating sub-questions, or producing multiple paraphrases whose results are merged. Multi-query retrieval and HyDE are common variants. It raises recall for short or ambiguous queries at the cost of extra retrieval calls.

HyDE

HyDE (Hypothetical Document Embeddings) is a query-expansion technique where the model first drafts a hypothetical answer to the question, then embeds that draft and retrieves real documents similar to it. The synthetic answer often sits closer in vector space to relevant passages than the terse original query. It can improve recall for under-specified questions.

Maximal Marginal Relevance (MMR)

Maximal Marginal Relevance is a selection strategy that picks results balancing relevance to the query against novelty relative to already-selected results, reducing redundancy in the returned set. It is useful when top vector hits are near-duplicates that waste the context window. A lambda parameter trades relevance against diversity.

Metadata Filtering

Metadata filtering restricts vector search to chunks matching structured attributes such as source, language, date, tenant, or access level, applied before or alongside the similarity search. It enforces correctness and security (for example tenant isolation) and narrows the search space. Combining filters with ANN is sometimes called filtered or hybrid-metadata search.

Multi-Tenancy

Multi-tenancy is the design of a RAG system to serve multiple isolated customers from shared infrastructure while guaranteeing that one tenant can never retrieve another's documents. It is usually enforced with a mandatory tenant filter on every query plus per-tenant namespaces or partitions. Getting isolation wrong is a data-leak class of bug.

Agentic RAG

Agentic RAG wraps retrieval in an agent loop where the model can decide whether to retrieve, reformulate the query, retrieve again, or call other tools, rather than retrieving exactly once. It handles multi-hop questions and ambiguous queries that single-shot RAG fails. The cost is added latency, complexity, and the need to bound the loop.

Multi-Hop Retrieval

Multi-hop retrieval answers questions that require chaining evidence from several documents, retrieving iteratively where each step's findings inform the next query. Single-pass retrieval often misses the bridging facts such questions need. Agentic loops and query decomposition are typical implementations.

Contextual Retrieval

Contextual retrieval is a technique that prepends a short, chunk-specific summary of its surrounding document to each chunk before embedding, so isolated passages retain the context they would otherwise lose. Introduced by Anthropic, it measurably reduces retrieval failures. It trades extra ingestion-time LLM cost for better recall.

GraphRAG

GraphRAG augments retrieval with a knowledge graph of entities and relationships extracted from the corpus, enabling queries that need global structure or multi-entity reasoning rather than isolated passages. It can answer "themes across the whole corpus" questions that flat vector search handles poorly. It adds graph-construction cost and complexity.

Prompt Injection

Prompt injection is an attack where malicious instructions embedded in retrieved or user-supplied content hijack the model into ignoring its system prompt or performing unintended actions. In RAG it is acute because untrusted documents enter the prompt as context. Defences include input/output filtering, privilege separation, and treating retrieved text as data, not instructions.

Token

A token is the unit a language model reads and generates, typically a sub-word fragment; English text averages roughly four characters per token. Token counts drive both cost and the context-window budget that RAG must fit its context into. Tokenisation differs per model, so the same text costs different token amounts across providers.

Temperature

Temperature is a decoding parameter that controls randomness in a model's output: lower values make it more deterministic and focused, higher values more varied and creative. Grounded RAG answers usually use a low temperature (near 0) to stay faithful and reproducible. It does not affect retrieval, only generation.

Caching

In RAG, caching stores reusable artefacts to cut cost and latency: embedding caches avoid re-embedding unchanged text, retrieval caches reuse results for repeated queries, and prompt/KV caching reuses a stable prompt prefix across calls. Each cache needs an invalidation strategy tied to its source's freshness. Prompt caching is especially effective when a large grounding context is reused.

Ground Truth

Ground truth is the human-verified reference answer (and optionally the reference context) for an evaluation question, against which a system's output is scored. It anchors metrics like context recall and answer correctness that need a known-correct target. Building a small, high-quality ground-truth set is the foundation of trustworthy RAG evaluation.