81 - What an AI Agent Actually Is (and When You Don't Need One)

An agent is a program in which a language model decides by itself which actions to take, in what order, and when to stop. The difference from an ordinary model call is not the power of the model: it is who holds the steering wheel. In a pipeline you write the path and the model fills in the blanks; in an agent the path is the model's choice, made at every turn on the basis of what it discovered on the previous one. That buys you flexibility and costs you predictability, money and testability — and most of the craft is knowing when that trade is worth taking.

The problem: three requests that look the same and are not

Look at these three, addressed to a system that has access to your document archive:

1. "Summarise this PDF in five lines."
2. "Summarise this PDF and translate it into English."
3. "Find in our contracts all the automatic-renewal clauses that expire in the next 90 days and tell me which ones must be cancelled."

The first is a single call. The second is a pipeline: two calls in a row, the output of the first goes into the second, and you decide the graph at writing time and it never changes. The third is different. You do not know in advance how many documents will have to be opened, you do not know whether a keyword search or a semantic one will be needed, you do not know whether after reading contract 7 you will have to go back to an attached table. The number of steps depends on what you find along the way. This is where an agent earns its keep.

The boundary is this, and it is worth memorising: if you can draw the graph of the calls before running, you do not need an agent. Write the pipeline, which is faster, cheaper, easier to test, and does not surprise you at three in the morning.

The loop, in a drawing

Before going down into the code, look at the shape. An agent is a closed ring with a single way out: the model observes the state, decides, acts through a tool, observes the result and goes back to deciding. It leaves only when it decides to speak instead of acting — or when it runs into a limit that you put there.

The minimum vocabulary, defined once

I will use these terms constantly in the rest of the module. Here I define them, so that no lesson has to take them for granted.

• Token. The unit in which a model counts text: a fragment of a word. The ratio between words and tokens changes from model to model and from language to language, so it is not estimated by eye: you measure it with the counter of the model you use. Tokens are paid for on the way in and on the way out, and they are the unit in which the whole cost of an agent is measured.
• Context (or context window). The maximum number of tokens the model can have in front of it in a single call: instructions, history, tool descriptions and the answer being produced, all together. It is a physical ceiling: past it, the endpoint rejects the request.
• Prompt. Everything you send to the model. Not only the user's question: also the system instructions and, which is easy to forget, the descriptions of the tools.
• Tool. A function of your program that you have made visible to the model by describing it with a name, an explanation and the schema of the arguments it accepts.
• Tool calling (or function calling). The mechanism by which the model, instead of answering in prose, emits a structured object of the kind "call read_stock with {"sku": "SKU-11"}". The model does not execute: it formulates a request that your code may decide to carry out.
• Turn (of the loop). One complete iteration: one call to the model plus the execution of any tools it asked for.
• Handoff. Passing control from one agent to another: from the next turn, the new agent answers. We cover it in the handoffs lesson.
• Guardrail. A check that runs alongside the agent and can stop it if something goes wrong — a filter on input or on output. We get there later in the module.
• Tracing. The structured recording of every step of a run, so that you can reconstruct it afterwards. Also later in the module.
• Hallucination. A statement by the model that is formally plausible and has no basis in the data it saw. In agents it shows up in its worst form: "I did X" when X was not done.

If you had to keep a single line of this glossary in mind, I would pick the one on tool calling: the model formulates a request, your code carries it out. Almost everything else (cost, security, testability) follows from that division of roles, and that is why token and context sit in the same list: they are the two quantities the loop consumes at every turn. Handoff, guardrail and tracing are extensions instead: you will meet them once the basic loop is familiar, and you do not need them to follow the rest of this lesson.

The analogy: the satnav and the driver

A pipeline is a printed itinerary: turn right at the third traffic light, then take the ring road, then exit at Casalecchio. If the ring road is closed, the sheet of paper is useless.

An agent is a driver with a satnav. You tell it the destination and the constraints ("avoid tolls, no more than two hours"), and at every junction it looks at the situation and chooses. It copes with the closed ring road. But it can also go round in circles, it can take a road that costs twice as much, and you cannot predict to the minute when it will arrive. A printed itinerary is stupid but verifiable; a driver is adaptive but needs supervising.

This image also explains why agents fail in odd ways: the typical failure is not an exception, it is "it arrived, but by the wrong route" or "it said it had arrived and it had not".

The mechanism: what really happens at each turn

Inside every agent framework, under the layers of abstraction, there is the same loop:

1. You send the model the instructions, the conversation history and the description of the available tools.
2. The model answers in one of two ways: with text for the user, or with one or more call requests to a tool, arguments included.
3. If it asked for a tool, your code (not the model) runs the function and puts the result back into the history.
4. You go back to step 1.
5. You leave when the model answers with text instead of asking for tools, or when you run into a limit you imposed yourself.

Two heavy consequences follow. The first: the model executes nothing, it asks. Every ability to act that an agent has, you gave it by writing a function and registering it. The second: the history grows at every turn, and at every turn you send all of it again. The cost of an agentic conversation grows much faster than linearly in the number of steps — a detail we see with numbers in the conversational memory lesson.

Steps 1 to 4 have a name in the literature: it is the skeleton of ReAct (Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models, arXiv:2210.03629), which interleaved explicit reasoning and action. In today's models the "reasoning" can stay internal to the model and the "acting" part goes through function calling, but the frame is the same one. For the wider theoretical picture, the dedicated agents series on this site covers it in fundamentals: OODA, ReAct and tool calling, while the formal definition of an agent in the classical literature (rationality, perception, environment) lives in what is an intelligent agent: definitions and taxonomy. I will not repeat them here: I start from what you need in order to write code.

Not two categories, but a scale of autonomy

So far I have set "pipeline" and "agent" against each other as if they were two boxes. It is a useful simplification to begin with and a wrong one if you stop there: in reality there is a scale, and the craft consists in placing yourself on the lowest rung that solves the problem. Each rung adds a capability and removes a guarantee, and the story of LLM applications that ended badly is largely the story of rungs climbed without need.

Rung 0 — a single call. One prompt, one answer. Summarise, translate, classify, rephrase. It is the most underrated rung: an enormous number of real problems live here, and here everything is predictable — one cost, one latency, one point of failure. If your problem sits at rung 0, anything more is dead weight.

Rung 1 — the fixed chain. Several calls in sequence, each with a different task, linked by you. Extract the data, then evaluate, then write the email. The graph is in the code and never changes. It is testable with ordinary assertions, the cost is known in advance, and it has a defect that is easy to forget: each step inherits the errors of the previous one without being able to correct them. If the extraction gets a number wrong, the email will repeat it with confidence.

Rung 2 — the router. A call whose only purpose is to classify, followed by a branch in your code. "What is this request about? billing / technical / sales", and then three different pipelines. Here the model makes one decision and you keep control of the flow. It is the rung that solves most of the cases people tackle with a full agent, and it costs a fraction.

Rung 3 — the agent with tools. The model decides, at every turn, whether to act and how. The number of steps is not known. It is the subject of this module, and it is needed when the path depends on what is discovered while running.

Rung 4 — several agents. Specialists that hand control to each other or call each other. It adds the ability to give different instructions and permissions to different parts of the system, and it adds a brand-new class of failures: loops between agents, context lost in the handoffs, multiplied costs. We discuss it in the lessons on handoffs and on orchestration.

The criterion for stopping is always the same: climb a rung only when you have seen, with your own data, that the rung below is not enough. "Not enough" means having collected real examples where it fails, not having imagined that it might fail. The difference between the two is a quarter of work.

Why now: what changed technically

It is worth understanding why this way of programming became practical only recently, because it also explains where it is still fragile. The idea is not new: the work of Yao and colleagues on ReAct (arXiv:2210.03629, October 2022) showed that alternating explicit reasoning steps and actions towards an external environment improved answers on tasks that require information not present in the model. A few months later Toolformer (Schick et al., arXiv:2302.04761) showed that a model can learn by itself when it pays to call an external tool, without human supervision on every example.

Both works, however, lived in a world where the only interface was text. The agent "called" a tool by writing a string in an agreed format, and the program recognised it with a regular expression. It worked in papers and broke in production for stupid reasons: a wrong quotation mark, a format changed halfway through, an omitted parameter. Three things changed the situation.

Native function calling. Models are trained on examples in which the request for a tool is a structured object, not prose — and the endpoint returns that object in a field separate from the text. The fragile parsing disappears. The error does not disappear (the model can ask for the wrong tool or fill a field badly), but it becomes a semantic error, not a syntactic one: checkable with a schema and with validation.

Large context windows. An agent's loop re-sends the whole history at every turn. With a window of a few thousand tokens an agent saturates the context after a few turns, because each turn adds tool results to the history. With much larger windows many more turns become possible, at a price: each turn sends the whole history again, and you see the bill in the lesson on memory.

Instruction adherence. It is the least visible part and the most decisive: an agent is governed by a text that describes constraints ("do not call this tool above 1000 euros"). If the model ignored it with appreciable frequency, the system would be unusable; if it follows it almost always, it is usable — provided you still have the checks in the code, because "almost always" is not "always" and never will be.

From this reconstruction you can see where the ground is still soft. The three advances concern the mechanics; none of the three gives guarantees about the substance. A model that formats a tool request perfectly may have chosen it for the wrong reasons, and a model that follows instructions may follow the wrong ones, that is, those that arrived inside the text it read from a tool. That is the theme of the lesson on real-world pitfalls, and it is not a model-version problem: it is structural.

The four minimum ingredients

Remove one and you no longer have an agent. The instructions are the mandate: goal, constraints, style, what not to do. They are underrated because "it's only text", yet with identical tools two sets of instructions produce agents that behave differently. The tool catalogue is the set of functions, each with a name, a description and an argument schema: the model chooses among these and only these. The state is the history of messages, of the tools called and of the results obtained; without state every turn starts from zero and the agent goes round in circles. The termination condition is in fact double: the "good" one (the model declares it has finished) and the "safety" one (maximum number of turns, token budget, time limit). The second is not optional, and I anticipate it in the code below before taking it apart in Anatomy of an agent.

The same task, two architectures

Let us compare them head to head. The task: given a product code, say whether it must be reordered. This requires reading the stock and the reorder threshold. As the model I use Ollama locally, so no keys and no spending are needed; with a managed endpoint the address and the message format change, and the format must be re-read in the provider's documentation. Careful: in this lesson the two blocks that call Ollama are patterns to adapt, not executed: on the machine where I checked them there is no model to query. The model llama3.1:8b is an example of a model that Ollama declares capable of tool calling; check on the model's page that yours is. The other blocks, the ones without network, I ran twice with identical output (Python 3.14.4).

Pipeline version — the graph is mine, the model does only the linguistic work. I read the data myself from the dictionary (step 1) and entrust only the sentence to the model (step 2): if the model gets it wrong, it gets the writing wrong, not the choice of which data to read. httpx.post with stream: False returns the whole answer in one go; raise_for_status() turns an HTTP error into an exception instead of leaving you an error JSON to interpret. Not executed.

import httpx

WAREHOUSE = {"SKU-11": {"stock": 4, "threshold": 10}}

def ask(prompt: str) -> str:
    r = httpx.post("http://localhost:11434/api/generate", timeout=120,
                   json={"model": "llama3.1:8b", "prompt": prompt, "stream": False})
    r.raise_for_status()
    return r.json()["response"]

def evaluate(sku: str) -> str:
    row = WAREHOUSE[sku]                        # step 1: I read the data
    return ask(                                 # step 2: the model writes the sentence
        f"Stock {row['stock']}, threshold {row['threshold']}. "
        "In one sentence: should it be reordered? Answer YES or NO and give the reason."
    )

print(evaluate("SKU-11"))

Two steps, always two, in this order. The number of calls is known before starting, so the cost is predictable. The weak point is rigidity: if the request becomes "check all the SKUs of supplier X and, where needed, first verify whether an order is already on its way", this function has to be rewritten.

Agent version — the graph is decided by the model, I only expose capabilities. The loop is written by hand on purpose, to show that there is no magic; in the next lessons an SDK does the same things with fewer lines. I isolated in call_model the only point that talks to the model: this way the loop can be tried without a model, by giving it a script, and the rest of the code stays identical when you connect it to a real model. Two details in the loop: a tool's exception becomes an {"error": ...} message returned to the model, which can correct itself; and the return message uses the tool_name field, which is the one described in the Ollama documentation on tool calling.

import json

WAREHOUSE = {"SKU-11": {"stock": 4, "threshold": 10, "incoming": 20}}

def read_stock(sku: str) -> dict:
    r = WAREHOUSE[sku]
    return {"sku": sku, "stock": r["stock"], "threshold": r["threshold"]}

def read_incoming_orders(sku: str) -> dict:
    return {"sku": sku, "incoming": WAREHOUSE[sku]["incoming"]}

TOOLS = {"read_stock": read_stock,
         "read_incoming_orders": read_incoming_orders}

def run(request: str, call_model, max_turns: int = 6) -> str:
    messages = [
        {"role": "system", "content": "Decide whether to reorder. Use the tools for data. "
                                      "Do not invent numbers. Finish with a single line."},
        {"role": "user", "content": request},
    ]
    for turn in range(1, max_turns + 1):
        msg = call_model(messages)                         # the only point that talks to the model
        messages.append(msg)
        calls = msg.get("tool_calls") or []
        if not calls:                                      # "good" termination
            print(f"turn {turn}: final answer")
            return msg.get("content", "")
        for c in calls:
            name = c["function"]["name"]
            args = c["function"]["arguments"]
            if isinstance(args, str):
                args = json.loads(args)
            try:
                outcome = TOOLS[name](**args)
            except Exception as e:                         # the error goes back to the model
                outcome = {"error": f"{type(e).__name__}: {e}"}
            print(f"turn {turn}: {name}({args}) -> {outcome}")
            messages.append({"role": "tool", "tool_name": name,
                             "content": json.dumps(outcome, ensure_ascii=False)})
    raise RuntimeError(f"no conclusion in {max_turns} turns")   # safety termination

Connecting the loop to Ollama takes a single function, the one that sends the history and returns the message field of the response. This too is a pattern to adapt, not executed. The SCHEMAS describe the two tools to the model (name, explanation, arguments); if the description is vague the model chooses badly, which is why the lesson on tools devotes space to it.

import httpx

SCHEMAS = [
    {"type": "function", "function": {
        "name": "read_stock",
        "description": "Current stock and reorder threshold of a SKU.",
        "parameters": {"type": "object", "required": ["sku"],
                       "properties": {"sku": {"type": "string"}}}}},
    {"type": "function", "function": {
        "name": "read_incoming_orders",
        "description": "Quantity already ordered and in transit for a SKU.",
        "parameters": {"type": "object", "required": ["sku"],
                       "properties": {"sku": {"type": "string"}}}}},
]

def call_ollama(messages: list) -> dict:
    r = httpx.post("http://localhost:11434/api/chat", timeout=180,
                   json={"model": "llama3.1:8b", "messages": messages,
                         "tools": SCHEMAS, "stream": False})
    r.raise_for_status()
    return r.json()["message"]

print(run("Should SKU-11 be reordered? Take into account what is already on its way.", call_ollama))

To try the loop without a model, I replace call_ollama with a script: a fake model that at every turn returns the next answer from a list. It does not prove that a real model behaves like that; it proves that your loop behaves well in the three cases that matter: a multi-turn path, a tool that raises an exception, and a model that never stops asking. This one was executed, twice, with identical output.

def fake_model(script):
    it = iter(script)
    return lambda messages: next(it)

def asks(name, **args):
    return {"role": "assistant", "content": "",
            "tool_calls": [{"function": {"name": name, "arguments": args}}]}

script = [
    asks("read_stock", sku="SKU-11"),
    asks("read_incoming_orders", sku="SKU-11"),
    {"role": "assistant", "content": "Do not reorder: 4 in stock + 20 incoming cover the threshold of 10."},
]
print(run("Should SKU-11 be reordered?", fake_model(script)))

script = [asks("read_stock", sku="SKU-99"),
          {"role": "assistant", "content": "SKU-99 does not exist: I cannot decide."}]
print(run("What about SKU-99?", fake_model(script)))

try:
    run("Spinning in circles", lambda m: asks("read_stock", sku="SKU-11"), max_turns=3)
except RuntimeError as e:
    print("RuntimeError:", e)

Output obtained:

# turn 1: read_stock({'sku': 'SKU-11'}) -> {'sku': 'SKU-11', 'stock': 4, 'threshold': 10}
# turn 2: read_incoming_orders({'sku': 'SKU-11'}) -> {'sku': 'SKU-11', 'incoming': 20}
# turn 3: final answer
# Do not reorder: 4 in stock + 20 incoming cover the threshold of 10.
# turn 1: read_stock({'sku': 'SKU-99'}) -> {'error': "KeyError: 'SKU-99'"}
# turn 2: final answer
# SKU-99 does not exist: I cannot decide.
# turn 1: read_stock({'sku': 'SKU-11'}) -> {'sku': 'SKU-11', 'stock': 4, 'threshold': 10}
# turn 2: read_stock({'sku': 'SKU-11'}) -> {'sku': 'SKU-11', 'stock': 4, 'threshold': 10}
# turn 3: read_stock({'sku': 'SKU-11'}) -> {'sku': 'SKU-11', 'stock': 4, 'threshold': 10}
# RuntimeError: no conclusion in 3 turns

Read the output as three stories. In the first the path has three turns: two reads and the answer, and the number of turns is decided by the script, not by your code. In the second the KeyError does not stop the program: it becomes a piece of data that the model reads and answers. In the third the model always asks for the same tool, and only the max_turns ceiling interrupts the loop. The cost of this flexibility is clear: compared with the two fixed steps of the pipeline the code is longer and the number of calls is no longer known in advance, because the model decides it (in the script, I do).

The decision criterion, in four questions

Before writing an agent, answer these. If the answer to the first is "yes", stop: you are done, write a pipeline.

1. Can I draw the graph of the calls before running? Yes → pipeline.
2. Does the number of steps depend on what I find along the way? No → pipeline, perhaps with a couple of if branches.
3. Are the actions reversible, or is there a human review? No to both → do not give autonomy over those actions: separate them and put them behind an approval.
4. Can I tell, by looking at an output, whether it is right? No → you have no evaluation criterion, and without one you will never know whether the agent works. Build the criterion first.

The first two questions decide whether you need an agent; the third decides where to put it, because autonomy over irreversible actions (sending money, deleting data) is a risk that no prompt eliminates; the fourth decides whether you will be able to notice that it works. It is the one most often ignored and the most expensive to ignore: an agent with no evaluation criterion is judged by feel, and feelings get worse exactly when the difficult cases increase.

The cost of the architectural choice, with numbers

Let us put the two approaches on the same problem and count the calls. The task: for a list of 200 SKUs, say which ones to reorder taking into account the orders in transit.

With the pipeline there is no call to the model in the normal case: the rule is stock + incoming < threshold, that is one line of Python executed 200 times, which on any machine is a matter of fractions of a millisecond (I did not time it: the point is the order of magnitude compared with a network round trip to a model). The model is needed only at the end, to write the summary: one call, a few hundred tokens. Total cost: one call.

With the agent, if you let it decide how to proceed, each SKU requires at least two reads and two or three model turns. Two hundred SKUs become 400 tool calls and, under those assumptions, 400 to 600 model calls, each with a history that keeps growing. These are counting assumptions, not measurements: the real number depends on the model and the prompt. The expected result is the same as the line of Python, with a cost and a latency orders of magnitude higher — and with the possibility that on some SKUs the model gets the arithmetic wrong.

This example is deliberately extreme, and serves to isolate the principle: an agent is not a way of executing rules, it is a way of deciding which rules to apply when you do not know in advance. As soon as the rule is known, the agent becomes an expensive and unreliable interpreter of something that your programming language executes perfectly.

The mixed form is often the right one, and it is worth spelling out because it is the answer to many real problems: the deterministic loop lives in the code, the agent handles the exceptions. You scan the 200 SKUs with the rule; the seven cases in which the rule does not decide (a supplier note in free text, a contradictory quantity, a SKU without a threshold) you hand to the agent, one by one. With these hypothetical numbers, seven runs instead of two hundred, on exactly the cases that require judgement.

The mixed form, in code

Since it is the form you would really use on a real problem, it is worth seeing it written out. The block is executable: the agent is a coroutine that you pass in, and here I use a fake one that always answers "to_check", to show the path without a model (executed twice, identical output). The deterministic rule decides the great majority of cases; the agent steps in only on the cases the rule declares ambiguous, and the reason for the ambiguity is passed to it explicitly.

import asyncio
from dataclasses import dataclass

@dataclass(frozen=True)          # immutable: nobody changes a row under your nose
class Row:
    sku: str
    stock: int | None
    threshold: int | None
    incoming: int
    supplier_note: str = ""

def decide_by_rule(r: Row) -> dict | None:
    """Return the decision if the rule is enough, otherwise None (case to escalate)."""
    if r.stock is None or r.threshold is None:
        return None                                    # missing data: judgement needed
    if r.supplier_note.strip():
        return None                                    # free text: reading needed
    covered = r.stock + r.incoming
    if covered < r.threshold:
        return {"sku": r.sku, "outcome": "reorder",
                "quantity": r.threshold * 2 - covered, "by": "rule"}
    return {"sku": r.sku, "outcome": "do_not_reorder", "quantity": 0, "by": "rule"}

async def evaluate_catalogue(rows: list[Row], agent, max_escalation: int = 20) -> dict:
    """`agent` is a coroutine: it receives the text of the case and returns a dict."""
    decided, to_escalate = [], []
    for r in rows:
        d = decide_by_rule(r)
        if d:
            decided.append(d)
        else:
            to_escalate.append(r)

    if len(to_escalate) > max_escalation:              # safety cap on cost
        return {"ok": False, "error": "too_many_ambiguous_cases",
                "count": len(to_escalate),
                "note": "check the data quality before spending on an agent"}

    async def one(r: Row) -> dict:
        outcome = await agent(
            f"SKU {r.sku}: stock={r.stock}, threshold={r.threshold}, "
            f"incoming={r.incoming}. Supplier note: {r.supplier_note!r}. "
            f"Decide whether to reorder and in what quantity."
        )
        return {**outcome, "by": "agent"}

    from_agent = await asyncio.gather(*(one(r) for r in to_escalate))
    return {"ok": True, "decisions": [*decided, *from_agent],
            "by_rule": len(decided), "by_agent": len(from_agent)}

async def fake_agent(text: str) -> dict:
    """Stands in for the model: always answers 'to_check', so the path is visible."""
    sku = text.split()[1].rstrip(":")
    return {"sku": sku, "outcome": "to_check", "quantity": 0}

ROWS = [
    Row("A1", 4, 10, 20),
    Row("A2", 4, 10, 0),
    Row("A3", None, 10, 0),
    Row("A4", 50, 10, 0, supplier_note="delivery suspended until January"),
]

async def main():
    out = await evaluate_catalogue(ROWS, fake_agent)
    for d in out["decisions"]:
        print(d)
    print(out["by_rule"], out["by_agent"])
    cap = await evaluate_catalogue(ROWS, fake_agent, max_escalation=1)
    print(cap)

asyncio.run(main())

Output obtained:

# {'sku': 'A1', 'outcome': 'do_not_reorder', 'quantity': 0, 'by': 'rule'}
# {'sku': 'A2', 'outcome': 'reorder', 'quantity': 16, 'by': 'rule'}
# {'sku': 'A3', 'outcome': 'to_check', 'quantity': 0, 'by': 'agent'}
# {'sku': 'A4', 'outcome': 'to_check', 'quantity': 0, 'by': 'agent'}
# 2 2
# {'ok': False, 'error': 'too_many_ambiguous_cases', 'count': 2, 'note': 'check the data quality before spending on an agent'}

Four details that make this code different from a demo. The function decide_by_rule returns None instead of guessing: it is the one that declares its own incompetence, and it declares it on two precise conditions (missing data, free text to be read). The max_escalation cap turns a data-quality problem into an explicit error instead of a bill: if half the catalogue is ambiguous, it is not a job for the agent, it is a job for whoever maintains the master data. The by field on every decision says whether the rule or the model took it, which is the information you will want when someone challenges a reorder. And the agent's runs go in parallel with gather, because they are independent — why, and the ways of getting it wrong, are in the asyncio lesson.

In the output, A1 and A2 are decided by the rule (A2: coverage 4 + 0 below the threshold of 10, quantity 10 × 2 − 4 = 16), while A3 for missing data and A4 for the free-text note go to the agent; with the cap lowered to 1 the function refuses to spend and says so. The quantity "threshold × 2 − coverage" is a rule invented for the example: yours depends on the supplier.

The ratio between the two final counters, by_rule and by_agent, is also the metric that tells you whether the architecture is healthy. If out of a thousand rows the agent touches ten, you are paying for intelligence only where it is needed. If it touches eight hundred, either the rule is badly written or the problem was not what you thought — and in both cases the answer is not a better prompt.

Diagnosis: recognising from the logs that you chose the wrong rung

Theory is of little use if you cannot check, with your own data, whether the choice was right. These four signals are read on a sample of a hundred runs and tell you everything.

1. The variance in the number of turns is low. If 95% of the runs make exactly three turns, you do not have an agent: you have a three-step chain with overhead on top. The autonomy you pay for you are not using. Go down a rung: put the three steps in the code.

2. The sequence of tools is always the same. A variant of the previous point, more specific and easier to measure: group the runs by the sequence of tools called. If three sequences cover 98% of the cases, those three are three pipelines, and the remaining 2% is the real job for an agent.

3. The distribution of turns has a long tail that touches the ceiling. If 4% of the runs end with max_turns exhausted, those runs are not "slow": they have failed, and the user probably received a truncated answer or an error. The long tail is where the cost lives and where the dissatisfaction lives, and you always look at it, because the average hides it.

4. The agent states actions it did not carry out. It is measured by comparing two things you already have: the actions declared in the final answer and the log of the tool calls actually executed. Every divergence is a case study. It is not solved with a better prompt — it is solved by never trusting the prose and checking against the log, and in some cases by taking away from the agent the ability to say it did things, replacing the final sentence with a text generated by your code on the basis of what really happened.

The first two signals say "you climbed too far", the third says "you underestimated the worst case", the fourth says "you are believing the narration instead of the facts". They are the four diagnoses that come back most often, and all four are read from data that the agent already produces — provided you record it. Hence the most boring and most profitable recommendation of this whole lesson: before putting an agent in front of users, make sure you record for each run the number of turns, the sequence of tools called, the exit reason and the tokens consumed. That is four fields. Without them you will not be able to answer any of the questions you will ask yourself the week after.

Trade-offs

Every advantage of the agent has a price, and it is worth stating them in pairs. Flexibility against predictability: the agent handles cases you did not foresee, and precisely for that reason you cannot foresee what it will do; you can only move the slider. Cost: a two-step pipeline makes two calls, an agent makes from two up to max_turns, each with a longer history than the previous one, and the worst case (not the average) is what you pay when things go wrong. Latency: sequential by construction, because each turn waits for the previous one; five turns are five round trips, which is why the lesson on asynchrony comes right after.

Testability: a pipeline is tested with assertions, an agent has different paths on the same input and needs statistical evaluation and run recording, not assertEqual. Attack surface: every tool is a door, and leaving to a model the choice of which ones to open, with arguments it decides on the basis of text that comes from outside, is a new and serious security problem.

The traps people really fall into

1. Using an agent for a deterministic problem. Symptom: in the logs, the agent always calls the same tools in the same order. That is a fixed graph with overhead on top. Turn it into a pipeline and you save money and headaches.
2. No turn limit. Two tools that pass the ball back and forth ("to answer I must search" → "the search is not enough, I must search better") produce loops that burn budget until someone notices. The limit goes in on the first day, not after the first incident.
3. Confusing "it answered" with "it did". An agent can declare "I sent the email" without having called the tool. The only truth is the log of the calls actually executed, not the final prose.
4. Too many tools. The larger the catalogue, the more the model errs in choosing, and meanwhile each description takes up context at every single turn. A few tools with sharp boundaries beat twenty overlapping ones.
5. Assuming the tool will be called correctly. The arguments are generated by a model: strings will arrive where you want integers, dates in three formats, nonexistent SKUs. Every tool is a system boundary and must be validated as such.

The five traps have a common root: treating the agent as if it were deterministic code, when it is a component that decides. The first two cost money (unnecessary autonomy, no brake), the third and the fifth cost correctness (believing the narration, trusting the arguments), and the fourth costs both. The remedy is the same in all cases: declare the boundaries in the code, record what happens and do not ask the model to be reliable in your place.

What you can do now

An agent is a loop in which a model chooses which function to call, reads the result and decides whether to continue. It is needed when the number and order of the steps depend on what is discovered while running; in all other cases a pipeline is better on every measurable aspect. You can decide with the four questions whether a problem requires an agent, place it on the lowest rung of the scale that solves it, and try an agentic loop without a model using a script. In the next lesson, Anatomy of an agent, we take the loop apart piece by piece — instructions, catalogue, state, termination — and see which design decisions hide in each.

In the series, the previous lesson is Deep Q-Learning: From Table to Neural Network, and Why Training Is Unstable.