01 - What is an intelligent agent: definitions and taxonomy

You have a script that reads a sensor, decides, and drives an actuator, and someone calls it an "agent". A colleague, looking at the same code, calls it "a function with an if". Both are right, because "agent" is not a property of the program: it is a way of describing what the program does with its environment. The question that really matters to whoever designs it is a different one: when it chooses an action, what information does it have in hand?

The short answer: an agent perceives the environment through sensors and acts on it through actuators, and agent types differ in what the program knows at the moment it decides. Here you measure it on a simulated vacuum cleaner with two rooms. One agent looks only at what it sees now, the other remembers what it has seen. The second wins almost always, but not always, and where it loses depends on how you wrote the score. That is why this article builds the experiment before giving the names.

What is meant by an agent

The operational definition you use here is the one in the two listings: an agent is a function from the history of percepts to an action, placed in an environment that reacts to the action and also changes on its own. Chapter 2 of Artificial Intelligence: A Modern Approach (Russell and Norvig, fourth US edition) is titled "Intelligent Agents" and opens with the section "Agents and Environments": the book's table of contents, published on the authors' website, confirms it. Of the chapter I read only the table of contents, not the text, so I do not quote sentences from the book in quotation marks.

The second classic source is the paper by Wooldridge and Jennings, "Intelligent agents: theory and practice", published in The Knowledge Engineering Review, volume 10, pages 115-152, in June 1995. The Crossref registry confirms title, authors, journal and pages; I cited the text of the paper without reading it, and I say so instead of attributing properties to it from memory. The two sources should not be merged into a single definition: the one used in the rest of the article is mine, and it holds for the two programs that follow.

Describing the task before writing the agent

Before writing a line of agent it is worth writing down what it is being asked to do, in four items. Section 2.3.1 of the chapter is titled "Specifying the task environment"; the acronym PEAS used to name these four items is the one you meet in textbooks, but I have not read the page and I do not present it as a quotation. For the simulated vacuum cleaner the four items are these.

• Performance measure: at each step you gain one point for every clean room, and you pay one point for every move.
• Environment: two rooms, A and B, each clean or dirty; at each step a clean room gets dirty with probability p_sporco (the dirt probability).
• Actuators: three actions, aspira (vacuum), sposta (move to the other room) and niente (do nothing).
• Sensors: the position and the dirt state of the room it is in, and nothing else.

The four items are not independent: the last one is what makes the problem interesting. With a sensor that saw both rooms the agent would read the full state and would not need to remember; seeing only the room it is in, it has to decide on partial knowledge. Almost all of the difference between agent types comes from this line.

The percept, decision, action cycle in code

The first block contains the two agents. They are functions that receive the percept, that is the pair position and dirt state, and return the name of an action. The reflex agent (agente_riflesso) has a single rule: if the room is dirty it vacuums, otherwise it moves. It has no memory, because the inner function saves nothing between one call and the next.

import random

QUADRATI = ("A", "B")


def altro(pos):
    return "B" if pos == "A" else "A"


def agente_riflesso():
    """Simple reflex agent: decides only on the current percept."""
    def agisci(percezione):
        pos, sporco = percezione
        return "aspira" if sporco else "sposta"
    return agisci


def agente_con_modello(validita):
    """Keeps a model of the room it cannot see: when it last saw it clean."""
    memoria = {"t": 0, "pulito_visto": {}}

    def agisci(percezione):
        pos, sporco = percezione
        memoria["t"] += 1
        if sporco:
            return "aspira"
        memoria["pulito_visto"][pos] = memoria["t"]
        visto = memoria["pulito_visto"].get(altro(pos))
        if visto is not None and memoria["t"] - visto < validita:
            return "niente"
        return "sposta"
    return agisci

The model-based agent (agente_con_modello) keeps a dictionary memoria (memory) with a step counter and the step at which it saw each room clean. If the room is dirty it vacuums; if it is clean it notes the step and checks when it saw the other one clean: if it saw it clean fewer than validita steps ago, it assumes it is still clean and does nothing, otherwise it goes to check. The validita parameter (validity) is the expiry of its knowledge. If you remove it, the agent never moves again after cleaning the two rooms, and you will see shortly that this costs dearly.

The second block is the environment and the loop. At each step the line with the comment "percept -> decision" passes to the agent only the position and the state of the current room, never the whole mondo (world) dictionary: it is the partial sensor of the task card above. Then the action modifies the world, the score adds the clean rooms and subtracts the cost of the move, and finally the environment gets rooms dirty on its own, with a random number generator initialised by the seed, so that two runs give the same numbers.

def esegui(fabbrica, p_sporco, passi, seme, costo_mossa=1):
    rng = random.Random(seme)
    mondo = {"A": True, "B": True}          # True = dirty
    pos = "A"
    agisci = fabbrica()
    punteggio = 0
    for _ in range(passi):
        azione = agisci((pos, mondo[pos]))   # percept -> decision
        if azione == "aspira":               # action
            mondo[pos] = False
        elif azione == "sposta":
            pos = altro(pos)
        puliti = sum(1 for q in QUADRATI if not mondo[q])
        punteggio += puliti - (costo_mossa if azione == "sposta" else 0)
        for q in QUADRATI:                   # the environment changes by itself
            if not mondo[q] and rng.random() < p_sporco:
                mondo[q] = True
    return punteggio


def media(fabbrica, p_sporco, passi=200, semi=100, costo_mossa=1):
    totale = sum(esegui(fabbrica, p_sporco, passi, s, costo_mossa) for s in range(semi))
    return totale / semi

Two agents in the same world: what the measurement says

Before the numbers, a check that does not depend on the simulation. If dirt never comes back (p_sporco = 0) and the agent starts with both rooms dirty, the score can be worked out by hand. The reflex agent does this: vacuums A (1 point), moves (1 minus 1, so 0), vacuums B (2 points), and from then on it earns 2 points and loses 1 at each move, that is 1 net point per step: the total over N steps is N. The model-based agent, with infinite expiry, after cleaning the two rooms never moves again and earns 2 points per step: the total is 2N - 3. With 200 steps that gives 200 and 397, and the assert statements in the block verify them. With p_sporco = 1 every percept is "dirty" and the two agents coincide: 200 and 200.

The third block computes the averages over 100 seeds (from 0 to 99), 200 steps, for six values of p_sporco, repeating everything with the move cost at 1 and at 0. The columns of the model-based agent use different expiries: 5, 20 and 60 steps, plus "never" (one million steps, that is no expiry). The program is the same one that produced the reported output, and it gives identical output in two consecutive runs.

if __name__ == "__main__":
    # Closed-form check (p = 0: no new dirt; p = 1: always)
    n = 200
    assert esegui(agente_riflesso, 0.0, n, 0) == n
    assert esegui(lambda: agente_con_modello(10**6), 0.0, n, 0) == 2 * n - 3
    assert esegui(agente_riflesso, 1.0, n, 0) == n
    assert esegui(lambda: agente_con_modello(10**6), 1.0, n, 0) == n

    assert esegui(agente_riflesso, 0.0, n, 0, costo_mossa=0) == 2 * n - 2
    assert esegui(lambda: agente_con_modello(10**6), 0.0, n, 0, costo_mossa=0) == 2 * n - 2
    print("closed-form checks: ok")

    for costo in (1, 0):
        print(f"cost of one move = {costo}")
        print("p_sporco  reflex    model(T=5)  model(T=20)  model(T=60)  model(never)")
        for p in (0.0, 0.01, 0.03, 0.1, 0.3, 1.0):
            r = media(agente_riflesso, p, costo_mossa=costo)
            m = [media(lambda t=t: agente_con_modello(t), p, costo_mossa=costo)
                 for t in (5, 20, 60, 10**6)]
            print(f"{p:<9} {r:<9.1f} {m[0]:<13.1f} {m[1]:<14.1f} {m[2]:<14.1f} {m[3]:.1f}")

# Output (two identical runs):
# closed-form checks: ok
# cost of one move = 1
# p_sporco  reflex    model(T=5)  model(T=20)  model(T=60)  model(never)
# 0.0       200.0     358.0         388.0          394.0          397.0
# 0.01      201.8     352.6         368.2          346.4          278.4
# 0.03      205.7     340.7         336.2          282.1          224.8
# 0.1       213.8     305.1         265.4          226.6          205.8
# 0.3       207.7     236.0         212.0          204.1          200.9
# 1.0       200.0     200.0         200.0          200.0          200.0
# cost of one move = 0
# p_sporco  reflex    model(T=5)  model(T=20)  model(T=60)  model(never)
# 0.0       398.0     398.0         398.0          398.0          398.0
# 0.01      396.1     392.6         378.2          350.4          279.4
# 0.03      392.0     380.7         346.2          286.1          225.8
# 0.1       374.6     344.5         275.4          230.6          206.8
# 0.3       311.9     272.9         222.0          208.1          201.9
# 1.0       200.0     200.0         200.0          200.0          200.0

Read the first table (move at 1 point). At p_sporco = 0.01 the agent with a 20-step expiry scores 368.2 against 201.8 for the reflex agent: remembering pays, because the reflex agent spends its time moving and paying. But the right expiry depends on the rate at which dirt comes back. At 0.01 the best of the four is 20 steps, from 0.03 upward it is 5. The "never" column falls to 278.4 at 0.01 and to 200.9 at 0.3: an agent that never forgets does not notice the dirt that arrives in the other room, and at 0.3 it does even worse than the reflex agent (207.7). With a 60-step expiry the same happens at 0.3: 204.1 against 207.7.

The second table (free moves) reverses the result. With p_sporco = 0 the four model-based agents and the reflex agent all score 398, because moving costs nothing and the rooms stay clean. As soon as dirt comes back, the reflex agent wins against all of them: 392.0 at 0.03 against 380.7 for the best model-based one. Neither agent is "more intelligent": the score rewards not moving only if moving costs. Changing the performance measure changes the winner, and that is why the task card comes before the agent.

Taxonomy as a question about knowledge

The names of the agent types answer a simple question: what is the action based on? The table of contents of chapter 2 of Russell and Norvig lists, in section 2.4, five structures: "Simple reflex agents", "Model-based reflex agents", "Goal-based agents", "Utility-based agents" and "Learning agents". I read the names in the table of contents; the descriptions in the table are my own summary, and the third column says what you have just run.

The table serves the problem you saw in the averages: I chose the expiry validita by hand, and the right choice changes with p_sporco. A utility-based agent would derive it by comparing expected scores; a learning agent could estimate the rate of dirt from experience. These are directions, not results: neither has been implemented here, and I do not tell you how much they would gain.

From the single agent to the multi-agent system

Put a second vacuum cleaner in the room. From the first one's point of view, the environment now contains another decision-maker: dirt no longer comes back only by chance, and a room that was clean a moment ago may be cleaned by someone else. I have not simulated this: I say it to clarify what changes, not to anticipate the numbers. The series goes on from exactly here: the BDI architecture gives a structure to the decision of a deliberative agent, the JADE framework brings several agents onto a platform, and FIPA-ACL communication says how they talk.

Where an agent is not needed and where the experiment misleads you

If the environment is fully observable and does not change while you decide, a well-written function is enough: in your world, with a sensor that saw both rooms, the memory of the model-based agent would not be needed. And the experiment itself has limits worth declaring, because the numbers in the table are true only inside this world.

• The expiry validita has to be tuned to the rate of the dirt: a long expiry at a high p_sporco loses against an agent that remembers nothing.
• The performance measure decides the winner: with free moves the reflex agent beats the model-based agent as soon as dirt comes back.
• The averages are over 100 seeds and over a world with two rooms and a single kind of event: they say nothing about behaviour with more rooms, noisy sensors or other agents.

The first line is a property of the agent with respect to the environment, the second is a property of the score, the third is a limit of the experiment. Mixing them leads to reading the table as "the agent with memory is better", a sentence that the data contradict in at least two cells and in one whole table.

Frequently asked questions

Is a thermostat an agent?

According to the operational definition of this article, yes: it has a sensor, an actuator and a condition-action rule, and it is a simple reflex agent. Whether it is interesting as an agent is another matter: with no memory and no goals of its own it has nothing more to say than its rule.

What is the concrete difference between a reactive agent and a model-based one?

The reactive one decides from the current percept alone; the model-based one has an internal state that it updates at every step. In the listings the difference is the memoria dictionary: with the right expiry it avoids useless moves, with the wrong one it stops noticing dirt.

Do you need an artificial intelligence model to have an agent?

No. The two agents are pure Python, with no external libraries and no models called: the rule of the reflex agent fits on one line. A model can be used to choose the action, but it is not the definition of an agent.

What you can do now

You can write the task card before the code and read in it where the difficulty comes from, in this case the partial sensor. You can turn an agent into a function from percept to action and measure it with a fixed-seed simulation, checked against a closed-form case. And you can avoid the wrong conclusion: no agent type wins in absolute terms, the one that wins is the one suited to the measure and to the pace of the environment. The rest of the series is in the series index.