LLM (Large Language Model)
A neural network trained on large text corpora to predict the next token, capable of generating and transforming natural language. LLMs are the reasoning core of every agent and RAG system. Examples include Claude, GPT, and Llama families.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant documents and adds them to the prompt so the model answers from supplied evidence rather than memory. It is the standard way to give a model fresh, proprietary, or citable knowledge without retraining.
MCP (Model Context Protocol)
An open standard from Anthropic for connecting LLM apps to tools, data, and prompts through a uniform client-server interface. It standardises integrations the way a common port standardises hardware.
ReAct (Reasoning and Acting)
A prompting pattern that interleaves reasoning traces with tool actions and observations in a loop. It is the conceptual basis for most tool-use agents.
CoT (Chain-of-Thought)
Prompting that elicits intermediate reasoning steps before the final answer, improving multi-step accuracy at the cost of extra tokens.
ToT (Tree-of-Thoughts)
An extension of chain-of-thought that explores multiple reasoning branches and selects the best, trading more compute for higher accuracy on hard search-like problems.
ANN (Approximate Nearest Neighbour)
A class of search algorithms that find vectors close to a query vector without an exhaustive scan, trading exactness for sub-linear latency. It is what makes vector search scale.
HNSW (Hierarchical Navigable Small World)
A graph-based ANN index offering a strong recall-to-latency trade-off, the default in many vector databases.
IVF (Inverted File Index)
An ANN index that partitions vectors into clusters and searches only the nearest clusters, often combined with product quantization (IVF-PQ) to save memory.
BM25
A classic lexical ranking function that scores documents by term frequency and inverse document frequency. It complements vector search in hybrid retrieval, excelling at exact keyword and rare-term matches.
RRF (Reciprocal Rank Fusion)
A method for merging results from multiple rankers (for example lexical and vector) by summing the reciprocal of each item's rank across lists. It is a simple, robust fusion strategy for hybrid search.
HITL (Human-in-the-Loop)
A design where a human reviews or approves an agent's actions at defined checkpoints, especially before irreversible operations. It trades autonomy for safety.
RLHF (Reinforcement Learning from Human Feedback)
A training method that aligns a model with human preferences via a learned reward model. It is why modern chat models follow instructions and refuse harmful requests.
DPO (Direct Preference Optimization)
A simpler alternative to RLHF that optimises a model directly on preference pairs without training a separate reward model. It has become a common alignment technique.
PEFT (Parameter-Efficient Fine-Tuning)
A family of fine-tuning methods that update only a small subset of parameters, reducing cost and storage. LoRA is the best-known example.
LoRA (Low-Rank Adaptation)
A PEFT technique that trains small low-rank matrices added to a frozen base model, making fine-tuning cheap and adapters portable.
TTFT (Time To First Token)
The latency from sending a request to receiving the first output token. It dominates perceived responsiveness in streaming chat interfaces.
TPM and RPM (Tokens / Requests Per Minute)
The two common rate-limit dimensions on LLM APIs: tokens per minute and requests per minute. Agents that loop and retry can exhaust these quickly, so back-off and batching matter.
kNN (k-Nearest Neighbours)
Retrieving the k closest items to a query in vector space; "top-k" in RAG refers to the k passages passed to the generator.
RAGAS
An open-source framework for evaluating RAG systems, providing reference-free metrics such as faithfulness, answer relevancy, and context recall/precision.
Faithfulness
An evaluation metric measuring whether an answer's claims are entailed by the provided context. It is the core trust metric for RAG, distinct from raw correctness.
Context Recall
A RAG metric measuring how much of the information needed to answer was actually retrieved into the context. Low context recall caps achievable answer quality.
Embedding Model
A model that maps text to fixed-length vectors capturing semantic meaning. It must be the same model (and dimensionality) at index time and query time.
Cross-Encoder
A model that scores a query and a document jointly in one pass, more accurate but slower than separate embeddings, used for reranking a candidate set.
Bi-Encoder
A model that encodes query and document independently into vectors compared by distance, fast enough to index a whole corpus, used for first-stage retrieval.
Top-k
The number of retrieved passages (or candidate tokens, depending on context) kept for the next stage. In RAG it is the count of chunks injected into the prompt.
Top-p (Nucleus Sampling)
A decoding parameter that samples from the smallest set of tokens whose cumulative probability exceeds p, controlling output diversity.
Tool Calling (Function Calling)
The API mechanism by which a model returns a structured request to invoke a developer-defined function, the foundation of agent capabilities.
Prompt Injection
An attack that smuggles adversarial instructions into user input or retrieved content to hijack an agent. Indirect injection hides the payload in fetched data.
Agent Loop
The repeating cycle of reason, act (tool call), observe (tool result) that an agent runs until a stop condition. Bounding it is essential to control cost.
Token
The sub-word unit a model reads and generates; pricing, context limits, and latency are all measured in tokens, not characters.
SLM (Small Language Model)
A smaller, cheaper, faster model used for narrow sub-tasks, routing, or on-device work where a frontier model would be overkill.