Agent
An agent is an LLM-driven system that pursues a goal across multiple steps by deciding which actions to take, executing them through tools, observing the results, and looping until the task is done or a stop condition is met. Unlike a single prompt-and-response call, an agent maintains state across iterations and controls its own flow. The distinguishing trait is that the model, not hardcoded logic, chooses the next step.
Agentic Workflow
An agentic workflow is an orchestration in which an LLM repeatedly reasons, acts, and observes inside a control loop rather than producing one final answer. Anthropic distinguishes "workflows", where the path is predefined in code, from "agents", where the model dynamically directs its own process. Most production systems are workflows with a few agentic steps, because predictable paths are easier to test and cheaper to run.
Action
An action is a concrete operation an agent emits to affect the world or gather information, typically a tool call with structured arguments. The agent proposes an action, an executor runs it, and the result is fed back as an observation. Constraining the action space (a finite, well-typed tool set) is the main lever for keeping agents reliable.
ReAct
ReAct (Reasoning + Acting) is a prompting pattern in which the model interleaves natural-language reasoning ("Thought") with tool calls ("Action") and tool results ("Observation") in a repeating loop. It improves task success over reason-only or act-only baselines by letting the model plan and gather evidence in the same trace. It is the conceptual ancestor of most modern tool-use loops.
Tool
A tool is an external capability exposed to an LLM as a callable function with a name, description, and a typed input schema, letting the model fetch data or take actions it cannot do from parameters alone. The model returns a structured request to invoke the tool; your code executes it and returns the result. Good tools are few, well-documented, and hard to misuse.
Tool Use (Function Calling)
Tool use, also called function calling, is the mechanism by which an LLM outputs a structured request to invoke a developer-defined function instead of plain text. The API returns the tool name and JSON arguments validated against the schema; the application runs the function and passes the result back for the next turn. It is the foundation of nearly all agent capabilities.
RAG (Retrieval-Augmented Generation)
RAG is a pattern that retrieves relevant documents from an external knowledge base and injects them into the prompt so the model answers from supplied evidence rather than parametric memory. It reduces hallucination, allows fresh and proprietary data, and makes answers citable. A RAG system has two halves: a retriever (search/embeddings) and a generator (the LLM conditioned on retrieved context).
Retriever
A retriever is the component of a RAG system that, given a query, returns the most relevant passages from a corpus, typically via vector similarity, lexical search, or a hybrid of both. Its quality sets the ceiling on answer quality: the generator cannot use evidence the retriever fails to surface. Retrieval is usually evaluated with recall@k and precision@k.
Embedding
An embedding is a dense numeric vector that represents the semantic meaning of text (or other data) so that similar meanings map to nearby points in vector space. Retrieval uses embeddings to find passages whose vectors are close to the query vector under a distance like cosine similarity. The embedding model and its dimensionality must be consistent across indexing and querying.
Vector Database
A vector database stores embeddings and supports fast approximate nearest-neighbour (ANN) search over them, often alongside metadata filters. Examples include pgvector, Pinecone, Weaviate, Qdrant, and Milvus. The defining feature is an ANN index (such as HNSW or IVF) that trades exact results for sub-linear query latency at scale.
HNSW
HNSW (Hierarchical Navigable Small World) is a graph-based approximate nearest-neighbour index that organises vectors into layered proximity graphs for fast logarithmic-time search. It offers an excellent recall-to-latency trade-off and is the default index in many vector stores. Its key parameters are M (graph connectivity) and efConstruction/efSearch (build and query effort).
Chunking
Chunking is the process of splitting source documents into passage-sized pieces before embedding, so each chunk is self-contained and small enough to retrieve precisely. Chunks that are too large dilute relevance; chunks that are too small lose context. Typical strategies split on structure (headings, paragraphs) with a token ceiling and optional overlap.
Context Window
The context window is the maximum number of tokens a model can attend to in a single request, covering the system prompt, conversation history, retrieved context, and the generated output. Exceeding it forces truncation or summarisation. Larger windows ease but do not eliminate context engineering, because relevant signal can still be lost among distractors.
Token
A token is the atomic unit a model reads and generates, usually a sub-word fragment produced by a tokenizer, roughly three-quarters of an English word on average. Pricing, context limits, and latency are all measured in tokens. Counting tokens (not characters) is essential for budgeting prompts and estimating cost.
Prompt Engineering
Prompt engineering is the practice of designing the instructions, examples, and structure of a prompt to reliably steer a model toward the desired behaviour. Techniques include role setting, clear task framing, few-shot examples, output-format constraints, and explicit reasoning requests. For agents it extends into tool descriptions and the loop's stop conditions.
System Prompt
A system prompt is the high-priority instruction block that defines an assistant's role, scope, rules, and output format for an entire conversation, separate from the user's turn-by-turn messages. It is where you place persona, guardrails, and grounding rules. Models weight it strongly, so it is the primary place to encode non-negotiable behaviour.
Few-Shot Prompting
Few-shot prompting supplies a handful of input/output examples in the prompt so the model infers the desired pattern without fine-tuning. It improves format adherence and task accuracy, especially for structured outputs. The examples should be diverse, correct, and representative of edge cases, because the model imitates them closely.
Chain-of-Thought
Chain-of-thought (CoT) prompting asks the model to produce intermediate reasoning steps before its final answer, which improves accuracy on multi-step problems such as math and logic. The trade-off is more tokens and latency. For user-facing products the reasoning is often generated but hidden, surfacing only the conclusion.
Hallucination
A hallucination is a model output that is fluent and confident but factually wrong or unsupported by any source. In agents and RAG it is the central reliability risk, typically mitigated by grounding answers in retrieved context, requiring citations, and allowing the model to say it does not know. It cannot be fully eliminated, only reduced and detected.
Grounding
Grounding is the practice of constraining a model's answer to supplied, verifiable evidence (retrieved passages, tool outputs, structured data) rather than parametric memory. A well-grounded system makes every factual claim traceable to a source and refuses when evidence is missing. Grounding is the primary defence against hallucination in RAG and agentic systems.
Faithfulness
Faithfulness measures whether a generated answer is entailed by the context it was given, with no claims that the context does not support. It is a core RAG evaluation metric (used by RAGAS and similar tools) and is distinct from correctness: an answer can be faithful to a wrong source. High faithfulness is what makes a RAG answer trustworthy and auditable.
MCP (Model Context Protocol)
The Model Context Protocol is an open standard, introduced by Anthropic, for connecting LLM applications to external tools, data sources, and prompts through a uniform client-server interface. An MCP server exposes resources, tools, and prompts; an MCP client (the host app) consumes them. It aims to be the "USB-C of AI integrations", replacing bespoke per-tool glue.
Function Schema
A function schema is the JSON Schema declaration of a tool's name, description, and typed parameters that the model uses to decide when and how to call it. Precise descriptions and constrained types (enums, required fields) dramatically improve call accuracy. The schema is the contract: the model reads it to plan, and your code validates against it.
Structured Output
Structured output is a model response constrained to a machine-parseable format, usually JSON conforming to a schema, so downstream code can consume it without brittle text parsing. Modern APIs enforce this via constrained decoding or a tool-call interface. It is essential for chaining steps, populating databases, and any non-conversational integration.
Orchestration
Orchestration is the layer that coordinates the steps, tools, models, and state of an agentic application, deciding what runs when and how results flow between stages. Frameworks like LangGraph, LlamaIndex, and the OpenAI Agents SDK provide orchestration primitives. The choice of orchestration shapes observability, error handling, and how easily the system can be tested.
Memory
Memory is the mechanism by which an agent retains information across turns or sessions beyond the immediate context window. Short-term memory is the conversation buffer; long-term memory persists facts to an external store (often a vector database) and retrieves them on demand. Without explicit memory, an agent is stateless between requests.
Planner
A planner is the component or prompt step that decomposes a high-level goal into an ordered set of sub-tasks before execution begins. Plan-and-execute architectures separate planning from acting so the plan can be inspected, cached, or revised. Planning improves reliability on long tasks but adds latency and a failure mode if the plan itself is wrong.
Multi-Agent System
A multi-agent system splits a problem across several specialised agents (for example a researcher, a coder, and a reviewer) that collaborate, often coordinated by an orchestrator agent. It can improve quality on decomposable tasks but multiplies cost, latency, and coordination complexity. The common guidance is to reach for it only when a single agent with good tools provably falls short.
Guardrails
Guardrails are the safety and policy controls placed around an LLM's inputs and outputs, such as input validation, content filters, allow-listed tools, output schema checks, and human approval gates for risky actions. They constrain the agent to acceptable behaviour even when the model errs. Defence in depth, applied at multiple layers, is more robust than a single check.
Human-in-the-Loop
Human-in-the-loop (HITL) is a design where a person reviews, approves, or corrects an agent's actions at defined checkpoints, especially before irreversible or high-impact operations. It trades autonomy for safety and is standard for agents that send messages, move money, or modify production systems. Well-placed approval gates are often the difference between a demo and a deployable agent.
Temperature
Temperature is a sampling parameter that controls the randomness of a model's output: lower values make it more deterministic and focused, higher values more varied and creative. For agents and extraction tasks a low temperature is usually preferred for consistency; for ideation a higher value helps. It interacts with top-p and should be tuned, not left at defaults blindly.
Top-p (Nucleus Sampling)
Top-p, or nucleus sampling, restricts token selection to the smallest set of candidates whose cumulative probability exceeds a threshold p, sampling only from that nucleus. It is an alternative to temperature for controlling diversity and is often used together with it. Lowering top-p tightens the output toward high-probability tokens.
Prompt Injection
Prompt injection is an attack in which adversarial instructions embedded in user input or retrieved content hijack an agent into ignoring its original instructions or misusing its tools. Indirect prompt injection, where the payload hides in a fetched web page or document, is especially dangerous for tool-using agents. Mitigations include input isolation, least-privilege tools, and human approval for sensitive actions.
Jailbreak
A jailbreak is an attempt to bypass a model's safety training or system instructions to elicit prohibited or out-of-scope behaviour, often via role-play framing or obfuscation. It differs from prompt injection in that it targets the model's policies directly rather than hijacking an application's instructions. Defences combine model-level safety training with application-level guardrails.
Fine-Tuning
Fine-tuning adapts a pretrained model to a specific task or style by further training it on a curated labeled dataset, adjusting its weights. It can improve format adherence, domain tone, and efficiency for narrow tasks, but is costlier and less flexible than prompting or RAG for injecting knowledge. The common rule is: try prompting and RAG first, fine-tune for behaviour, not for facts.
RLHF
RLHF (Reinforcement Learning from Human Feedback) is a training method that aligns a model with human preferences by training a reward model on human comparisons of outputs, then optimising the policy against it. It is a key reason modern chat models follow instructions and refuse harmful requests. It shapes default behaviour but does not replace application-level grounding and guardrails.
Evaluation (Evals)
Evaluation is the systematic measurement of an LLM system's quality against a fixed test set, using metrics such as faithfulness, answer relevancy, context recall, and task success. For agents, evals also cover tool-call correctness and end-to-end completion. Treating evals as a regression gate in CI is the discipline that turns prompt tweaking into engineering.
LLM-as-a-Judge
LLM-as-a-judge is an evaluation technique that uses a capable model to score or compare outputs against a rubric or reference, scaling assessment beyond manual review. It is useful for subjective qualities like helpfulness and faithfulness but must be calibrated against human labels and watched for biases (such as preferring longer or first-listed answers). It is a tool for scale, not a replacement for ground truth.
Reranking
Reranking is a second retrieval stage that reorders an initial candidate set with a more expensive, more accurate model (typically a cross-encoder) to push the best passages to the top. It improves precision@k without the cost of running the heavy model over the whole corpus. The common pattern is fast ANN retrieval of many candidates, then rerank to a small final set.
Hybrid Search
Hybrid search combines lexical retrieval (keyword/BM25) with dense vector retrieval and fuses the scores, capturing both exact-term matches and semantic similarity. It outperforms either method alone on heterogeneous queries, especially those with rare terms, codes, or names that embeddings handle poorly. Score fusion methods include Reciprocal Rank Fusion (RRF).
Streaming
Streaming is the delivery of a model's output token-by-token as it is generated, rather than waiting for the full response. It dramatically improves perceived latency in chat UIs and lets agents act on partial output. It complicates tool-call handling and error recovery, so streaming pipelines need careful state management.
Latency
Latency is the time between a request and a usable response, commonly split into time-to-first-token and total generation time. For agents it compounds across every loop iteration and tool call, so a five-step agent can feel slow even with a fast model. Reducing loop count, parallelising tool calls, and using smaller models for sub-tasks are the main levers.
Prompt Caching
Prompt caching reuses the computed representation of a static prompt prefix (such as a long system prompt or few-shot block) across requests, cutting cost and latency for the repeated portion. It is offered by major APIs and is especially valuable for agents that resend a large fixed context each turn. The cacheable part must be a stable prefix placed before the variable content.
Idempotency
Idempotency is the property that performing an operation multiple times has the same effect as performing it once. In agents it is critical because retries, loops, and at-least-once execution can replay a tool call; an idempotency key lets the executor deduplicate side effects. Designing tools to be idempotent is what makes agent retries safe.
Token Budget
A token budget is the explicit allocation of the context window across system prompt, history, retrieved context, and expected output, planned so the request fits within the window with headroom. Managing it involves trimming history, capping retrieved chunks, and summarising older turns. Without a budget, long-running agents silently degrade as context overflows.
Context Engineering
Context engineering is the discipline of assembling exactly the right information into a model's context window for a given step: the relevant retrieved passages, tool results, memory, and instructions, and nothing that distracts. It generalises prompt engineering to the full, dynamically assembled context of an agent. Good context engineering is often the highest-leverage fix for agent quality.
Stop Condition
A stop condition is the rule that ends an agent's loop, such as the model emitting a final answer, reaching a maximum number of iterations, exhausting a token or cost budget, or hitting an error threshold. Robust agents always bound their loops to prevent runaway cost and infinite cycles. A missing or weak stop condition is a common cause of expensive failures.
Cosine Similarity
Cosine similarity measures the closeness of two vectors by the cosine of the angle between them, ignoring magnitude, with 1 meaning identical direction and 0 meaning orthogonal. It is the most common distance metric for comparing text embeddings in retrieval. Many vector stores normalise vectors so that cosine similarity and dot product are equivalent.