LLM (Large Language Model)
A neural network, typically a Transformer, trained on large text corpora to predict the next token, which gives it broad language and reasoning capability. LLMs are the substrate that prompt engineering steers. Examples include the Claude, GPT, and Gemini model families.
CoT (Chain-of-Thought)
Shorthand for prompting the model to show intermediate reasoning steps before the final answer. Used to improve performance on multi-step reasoning tasks. Triggered by phrases like "think step by step" or by reasoning-style few-shot examples.
RAG (Retrieval-Augmented Generation)
An architecture that retrieves relevant external documents at query time and injects them into the prompt so the answer is grounded in current, authoritative data. It reduces hallucination and decouples knowledge from model weights. The standard pattern for question answering over private or fresh data.
ICL (In-Context Learning)
The ability of an LLM to learn a task from instructions or examples provided in the prompt at inference time, with no weight updates. It is the mechanism behind zero-shot and few-shot prompting. The adaptation lasts only for the current request.
ReAct (Reason + Act)
An agent prompting pattern that interleaves reasoning steps with tool actions and observations in a loop. It lets a model plan, gather information, and self-correct. Widely used as a basis for tool-using agents.
SFT (Supervised Fine-Tuning)
Training a base model on curated input/output pairs to specialize its behavior, often a step before preference-based alignment. It changes model weights, unlike prompting. Prompt engineering and SFT are complementary tools for steering behavior.
RLHF (Reinforcement Learning from Human Feedback)
An alignment technique that trains a reward model from human preference comparisons and then optimizes the LLM against it. It is a major reason instruction-tuned models follow prompts well. RLHF shapes the defaults that prompts then refine.
RAFT / Few-Shot
"Few-shot" denotes including a handful of examples in the prompt to demonstrate the desired pattern. The count is often written as N-shot (one-shot, five-shot). Examples should be correct, diverse, and cover edge cases.
Top-p
A sampling parameter (nucleus sampling) that limits token choice to the smallest set whose cumulative probability reaches p. It adapts the candidate pool to the model's confidence. Tune either top-p or temperature, not both.
Top-k
A sampling parameter that limits token choice to the k most probable candidates. Smaller k yields more conservative output. A simpler alternative to nucleus sampling.
MCP (Model Context Protocol)
An open standard introduced by Anthropic for connecting LLM applications to external tools, data sources, and prompts through a uniform client/server interface. It standardizes how context and tools are exposed to a model, replacing bespoke per-integration glue. MCP servers expose resources, tools, and prompt templates.
TPS / TPM (Tokens Per Second / Per Minute)
TPS measures generation throughput (output tokens per second), a key latency metric for streaming. TPM (tokens per minute) is a common rate-limit unit on hosted APIs. Both are budgeted in tokens, the universal currency of LLM serving.
TTFT (Time To First Token)
The latency from sending a request to receiving the first output token, the dominant felt-latency metric for interactive use. Prompt caching and shorter prompts reduce TTFT. It is distinct from total generation time, which depends on output length and throughput.
KV Cache (Key-Value Cache)
The cache of attention keys and values for already-processed tokens that lets a model generate each new token without recomputing the whole sequence. It is what makes prompt caching and fast autoregressive decoding possible. Reusing a stable prefix's KV cache is the basis of prompt caching.
OWASP LLM Top 10
A community-maintained list of the most critical security risks for LLM applications, with prompt injection ranked first (LLM01). It is the standard reference for threat-modeling LLM systems. It covers injection, insecure output handling, data poisoning, and more.
PII (Personally Identifiable Information)
Data that can identify an individual, which prompts and logs may inadvertently contain or leak. Handling PII safely (redaction, minimization, access control) is a compliance requirement for LLM applications. Prompt design should avoid sending unnecessary PII to the model.
DPO (Direct Preference Optimization)
An alignment method that optimizes a model directly on preference pairs without training a separate reward model, simplifying the RLHF pipeline. Like RLHF, it shapes model defaults rather than per-request behavior. Relevant to prompt engineers as context for why models behave as they do.
Token Budget
The total number of tokens available for a request, shared across system prompt, history, retrieved context, user input, and output. Managing this budget is central to context engineering. Exceeding it forces truncation or request failure.
Prefill
Supplying the beginning of the assistant's response to constrain its format or content, for example starting a JSON object with an opening brace. Prefilling steers output and suppresses preambles. It is a reliable formatting technique on chat APIs that support an assistant turn.
SLM (Small Language Model)
A compact LLM optimized for low latency, low cost, or on-device deployment, often used for routing, classification, or simple extraction. SLMs let cost-aware pipelines reserve large models for hard steps. Prompt techniques transfer but capacity limits demand simpler, more explicit prompts.
Hallucination Rate
A measured proportion of outputs containing unsupported or fabricated claims, evaluated against ground truth or provided context. It is a primary quality metric for factual and RAG systems. Grounding and abstention instructions reduce it.
Top-of-Funnel Prompt
Informal term for a broad, exploratory prompt used early in iteration before constraints and format are pinned down. It contrasts with a hardened production prompt. The progression from exploratory to hardened prompt mirrors prototype-to-production.
NLI (Natural Language Inference)
The task of deciding whether a hypothesis is entailed by, contradicts, or is neutral to a premise. NLI models are used to automate faithfulness checking in RAG by testing whether an answer is entailed by its context. It underpins automated grounding evaluation.
BLEU / ROUGE
Reference-based text-overlap metrics (BLEU for translation, ROUGE for summarization) that score generated text against reference text by n-gram overlap. They are cheap but correlate weakly with quality on open-ended tasks, where LLM-as-a-judge or human eval is preferred. Still useful as fast regression signals.
Context Rot
Informal term for the degradation in answer quality as irrelevant or excessive content accumulates in the context window, diluting the signal the model needs. It motivates pruning history and retrieving fewer, higher-quality passages. The fix is curation, not just a bigger window.
ANN (Approximate Nearest Neighbor)
A class of algorithms (such as HNSW) that find vectors close to a query vector quickly without an exact exhaustive scan, enabling fast semantic search at scale. ANN indexes power the retrieval step of RAG. They trade a small recall loss for large speed gains.
HNSW (Hierarchical Navigable Small World)
A graph-based ANN index widely used in vector databases for fast, high-recall similarity search. It is a common default for RAG retrieval backends. It balances query speed, recall, and memory through tunable parameters.
P50 / P95 / P99 Latency
Percentile latency metrics: P50 is the median, P95 and P99 capture the slow tail experienced by a minority of requests. Tail latency often dominates user experience in LLM apps and is targeted with caching, smaller prompts, and streaming. SLOs are typically set on P95 or P99.
Drift
The gradual divergence between a deployed prompt's behavior and current requirements, caused by model updates, changing data, or evolving needs. Evals run on a fixed golden set detect drift across model or prompt changes. Versioning prompts and pinning models reduce surprise drift.