Agent
An agent is an LLM-driven system that pursues a goal over multiple steps by deciding which actions or tools to invoke, observing the results, and looping until the task is done or a stop condition is met. Unlike a single prompt, an agent maintains state across turns and makes control-flow decisions itself. Anthropic distinguishes agents (the model directs its own process) from workflows (the orchestration is fixed in code).
Chain-of-Thought (CoT)
Chain-of-thought is a prompting technique that asks the model to produce intermediate reasoning steps before its final answer, which improves accuracy on arithmetic, logic, and multi-step problems. It is triggered by instructions such as "think step by step" or by few-shot examples that show worked reasoning. The reasoning trace is a means to a better answer, not necessarily a faithful account of the model's internal computation.
Context Window
The context window is the maximum number of tokens a model can attend to at once, spanning the system prompt, conversation history, retrieved documents, the user message, and the generated output. When input plus output would exceed this limit, the request fails or earlier content must be truncated. Effective context engineering treats the window as a scarce budget to be allocated deliberately.
Few-Shot Prompting
Few-shot prompting includes a small number of input/output examples in the prompt so the model can infer the desired pattern, format, or labeling scheme by analogy. It typically outperforms zero-shot for tasks with a specific output structure or domain convention. The examples should be diverse, correct, and representative of the edge cases you care about.
Zero-Shot Prompting
Zero-shot prompting asks the model to perform a task using only an instruction, with no worked examples. It works well for tasks the model has seen extensively during training and keeps prompts short, but it is less reliable for unusual formats or domain-specific conventions. When zero-shot output is inconsistent, adding a few examples (few-shot) is the standard next step.
Prompt
A prompt is the full input given to a language model for a single generation, including any system instructions, conversation history, retrieved context, and the user's request. Everything the model sees before it generates is part of the prompt and competes for the same context budget. Prompt engineering is the practice of structuring this input to make the desired output reliable.
System Prompt
A system prompt is a top-level instruction that sets the model's role, constraints, tone, and operating rules for the entire conversation, separate from individual user turns. It is the most durable place to encode behavior such as persona, output format, and safety guardrails. With the Anthropic API the system prompt is passed in the dedicated system parameter rather than as a message.
Temperature
Temperature is a decoding parameter that scales the randomness of token sampling: lower values make output more deterministic and focused, higher values make it more varied and creative. For extraction, classification, and factual tasks a low temperature is preferred; for brainstorming or creative writing a higher value helps. It is one of several sampling controls, alongside top-p and top-k.
Top-p (Nucleus Sampling)
Top-p, or nucleus sampling, restricts token selection to the smallest set of candidates whose cumulative probability reaches the threshold p, sampling only from that nucleus. It adapts the candidate pool to the model's confidence at each step, unlike top-k which fixes the count. Vendors generally recommend tuning either temperature or top-p, not both at once.
Top-k Sampling
Top-k sampling limits the next-token choice to the k most probable candidates and renormalizes over them before sampling. A small k makes output more conservative; a large k allows more diversity. It is a simpler alternative to nucleus (top-p) sampling and is often left at its default when temperature is the primary control.
Token
A token is the atomic unit a model reads and generates, typically a sub-word fragment produced by the tokenizer; English text averages roughly four characters per token. Pricing, context limits, and latency are all measured in tokens, so token efficiency is a practical concern in prompt design. The same string can tokenize differently across model families.
Tokenizer
A tokenizer is the component that converts text into the integer token IDs a model consumes and converts generated IDs back into text. Different model families use different tokenizers (for example byte-pair encoding variants), so token counts are not directly comparable across vendors. Understanding tokenization explains quirks such as why some characters or numbers behave unexpectedly.
Retrieval-Augmented Generation (RAG)
Retrieval-augmented generation supplies a model with relevant external documents retrieved at query time so its answer is grounded in current, authoritative data rather than only parametric memory. A typical pipeline embeds and indexes a corpus, retrieves the top-k passages for a query, and injects them into the prompt as context. RAG reduces hallucination and lets you update knowledge without retraining.
Grounding
Grounding is constraining a model's response to verifiable evidence supplied in the prompt, such as retrieved passages or tool outputs, so claims can be traced to a source. A well-grounded prompt instructs the model to answer only from the provided context and to abstain when the context is insufficient. Grounding is the core defense against hallucination in RAG systems.
Hallucination
A hallucination is model output that is fluent and confident but factually unsupported or fabricated, such as an invented citation, API, or statistic. It arises because the model optimizes for plausible continuations, not truth. Mitigations include grounding in retrieved context, instructing the model to say "I don't know," and verifying claims against sources.
Tool Use (Function Calling)
Tool use, also called function calling, lets a model request a defined external operation, such as a search, calculation, or API call, by emitting a structured invocation that your code executes and returns. The model is given tool schemas describing names, descriptions, and input parameters, and chooses when to call them. This is the mechanism that turns a text generator into an agent that can act.
Structured Output
Structured output is generation constrained to a machine-readable format such as JSON conforming to a specified schema, so downstream code can parse it reliably. It is achieved through schema-guided decoding, tool/function definitions, or explicit format instructions plus validation. Constraining the format removes brittle text parsing and is essential for production pipelines.
Prompt Template
A prompt template is a reusable prompt with named placeholders that are filled with runtime values, separating fixed instructions from variable input. Templates make prompts version-controllable, testable, and consistent across an application. Libraries and frameworks provide templating with variable substitution and validation.
Prompt Injection
Prompt injection is an attack in which adversarial instructions embedded in user input or in retrieved/external content cause the model to ignore its original instructions or perform unintended actions. Indirect prompt injection hides the payload in data the model later reads, such as a web page or document. It is the LLM analogue of injection attacks and the top risk in the OWASP Top 10 for LLM Applications.
Jailbreak
A jailbreak is a class of adversarial prompts crafted to bypass a model's safety training and elicit content the model is meant to refuse, often via role-play framing, obfuscation, or many-shot conditioning. Jailbreaks differ from prompt injection in intent and target: they attack the model's policy rather than an application's instructions. Defenses include refusal training, input/output filtering, and constitutional methods.
Guardrails
Guardrails are the controls that keep a model's inputs and outputs within acceptable bounds, including content filters, schema validation, allow/deny lists, refusal rules in the system prompt, and post-generation checks. They are layered: instruction-level guardrails in the prompt plus programmatic validation in code. Robust systems never trust the model output alone.
In-Context Learning
In-context learning is the model's ability to adapt to a new task from examples or instructions given in the prompt at inference time, without any weight updates. It is what makes few-shot and zero-shot prompting work. The model generalizes from the demonstrated pattern for the duration of that request only.
Prompt Chaining
Prompt chaining decomposes a complex task into a sequence of simpler prompts where each step's output feeds the next, improving reliability and making each stage testable. It trades a single hard prompt for several easy ones and gives natural checkpoints for validation. It is a core building block of workflows, distinct from a single mega-prompt.
ReAct
ReAct (Reason + Act) is an agent pattern that interleaves reasoning traces with tool actions: the model thinks, takes an action, observes the result, and repeats. Interleaving reasoning with acting lets the model plan, gather information, and self-correct within a loop. It underpins many tool-using agent implementations.
Self-Consistency
Self-consistency is a decoding strategy that samples multiple chain-of-thought reasoning paths at non-zero temperature and selects the most frequent final answer by majority vote. It improves accuracy on reasoning tasks over a single greedy decode by marginalizing over diverse reasoning paths. The cost is running several generations per query.
Reasoning Model
A reasoning model is one trained or configured to spend additional inference-time computation producing an extended internal chain of thought before answering, improving performance on hard math, coding, and planning. Such models expose controls for reasoning effort or budget. They trade higher latency and token cost for accuracy on complex problems.
Extended Thinking
Extended thinking is Anthropic's feature that lets a Claude model produce explicit internal reasoning before its final response, with a configurable token budget. It improves results on complex analysis, math, and multi-step tasks at the cost of additional latency and tokens. The thinking content is returned in dedicated blocks distinct from the final answer.
Prompt Caching
Prompt caching stores the processed representation of a stable prompt prefix so repeated requests sharing that prefix skip recomputation, cutting latency and cost. It is ideal when a long system prompt, schema, or document is reused across many calls. With the Anthropic API you mark cache breakpoints with cache_control on the relevant content blocks.
Context Engineering
Context engineering is the discipline of curating exactly what enters the context window at each step - instructions, examples, retrieved data, tool results, and history - to maximize useful signal within the token budget. It broadens prompt engineering from wording a single instruction to managing the whole information state of an agent. The goal is the smallest set of high-signal tokens that yields the desired behavior.
Role Prompting
Role prompting assigns the model a persona or expertise ("You are a senior security reviewer") to shape tone, vocabulary, and the standards it applies. It is a lightweight way to steer style and rigor and is usually placed in the system prompt. Its effect is on framing and emphasis rather than on granting new knowledge.
Delimiters
Delimiters are explicit markers - such as XML-style tags, triple backticks, or headings - that separate distinct parts of a prompt like instructions, context, and user input. They reduce ambiguity about what is data versus what is an instruction and help defend against prompt injection. Anthropic recommends XML-style tags such as
Output Indicator
An output indicator is a cue at the end of a prompt that signals where and how the model should begin its answer, such as a label, a partial JSON opening, or an assistant-turn prefix. Prefilling the start of the response constrains format and reduces preamble. It is a simple, reliable way to enforce a structure.
Negative Prompting
Negative prompting tells the model what not to do or include, such as "do not invent citations" or "do not include a preamble." Positive instructions ("answer only from the context") are generally more reliable than prohibitions, so negatives work best as a complement. Overloading a prompt with prohibitions can degrade performance.
Meta-Prompting
Meta-prompting uses a model to write, critique, or improve prompts rather than to solve the end task directly. A meta-prompt might ask the model to rewrite an instruction for clarity, generate few-shot examples, or diagnose why a prompt fails. It is a practical way to iterate on prompts at scale.
Prompt Optimization
Prompt optimization is the systematic, often automated, search for prompt wording, examples, or structure that maximizes a measurable metric on an evaluation set. Approaches range from manual A/B testing to algorithmic methods that mutate and score candidate prompts. It depends on having a reliable eval to optimize against.
Eval (Evaluation)
An eval is a dataset of inputs paired with expected outputs or graded criteria, used to measure prompt or model quality objectively rather than by eyeballing. Evals turn "this feels better" into a number you can track across changes and gate releases on. They are the foundation of disciplined prompt engineering.
LLM-as-a-Judge
LLM-as-a-judge uses a language model to score or compare other models' outputs against a rubric, enabling automated evaluation of open-ended responses that lack a single correct answer. It scales human-like judgment but inherits the judge's biases (such as position or verbosity bias), so it must be calibrated against human labels. It is widely used in eval pipelines and preference data collection.
Faithfulness
Faithfulness measures whether a generated answer is supported by the provided context, with no claims that the context does not entail. In RAG evaluation it is a core metric, typically scored by checking each claim against the retrieved passages. High faithfulness means low hallucination relative to the supplied evidence.
Embedding
An embedding is a dense vector that represents the meaning of a piece of text so that semantically similar texts lie close together in vector space. Embeddings power semantic search and the retrieval step of RAG by enabling nearest-neighbor lookup. The vector dimension and model determine quality and must match between indexing and querying.
Semantic Search
Semantic search retrieves results by meaning rather than exact keywords, comparing the embedding of a query to embeddings of indexed documents and returning the nearest neighbors. It complements lexical (keyword) search, which excels at exact terms and rare tokens. Hybrid search fuses both for better recall.
Chunking
Chunking splits source documents into passages small enough to embed and retrieve precisely while remaining self-contained enough to be meaningful out of context. Chunk size and overlap trade retrieval precision against context completeness; passages that are too large dilute relevance. Chunking strategy strongly affects RAG quality.
Stop Sequence
A stop sequence is a string that, when generated, causes the model to halt output immediately, used to bound responses or terminate a structured format cleanly. It prevents the model from running past the intended answer, for example stopping at a delimiter between turns. It is configured per request alongside max tokens.
Max Tokens
Max tokens is the cap on how many tokens a single generation may produce, bounding cost, latency, and the length of the answer. Setting it too low truncates the response mid-thought; setting it too high wastes budget. It limits only the output, separate from the overall context window.
System of Record
In a RAG or agent application, the system of record is the authoritative source whose data the model must reflect, as opposed to the model's parametric memory. Designing prompts to defer to the system of record - by grounding answers in retrieved data and citing it - keeps outputs trustworthy and auditable. Conflicts between model memory and the record must always resolve to the record.
Multi-Shot / Many-Shot
Multi-shot or many-shot prompting extends few-shot to a large number of in-context examples, which can substantially improve accuracy on some tasks when the context window is large enough. Returns diminish and cost rises as examples are added, so the count is tuned empirically. Many-shot can also be a vector for adversarial conditioning, which is a security consideration.
Constitutional AI
Constitutional AI is Anthropic's training method that uses a written set of principles (a "constitution") to have a model critique and revise its own outputs, reducing reliance on large volumes of human-labeled harmful examples. It shapes the model's default behavior and refusals. For prompt engineers it means base models already carry strong, principled defaults that prompts build on rather than replace.
Idempotent Prompt
An idempotent prompt produces equivalent results when re-run on the same input, a property aided by low temperature, fixed examples, and explicit output schemas. Idempotency matters for reproducible pipelines, caching, and testing. Non-determinism from sampling is the main obstacle and is reduced, not eliminated, by deterministic decoding settings.