87 - Conversational Memory: Sessions, Context Window and Summarisation

A language model remembers nothing: every call is stateless. The "memory" of an assistant is a list of messages that you send back to the model every time, in full. Three facts follow from this single sentence, and they govern the design of any conversational agent: the cost grows quadratically with the number of turns, the context window is a physical ceiling, and everything you want it to "remember" beyond that ceiling has to be rebuilt - by truncation, by summary or by fetching it back from an archive.

What I ran, and what I did not. Code tested on 2026-11-12 with openai-agents 0.24.0 and Python 3.14.4, in a temporary virtual environment outside the repository. No model was called, not even Ollama: the model is fake and answers with a script I wrote, through a simulated HTTP transport (no network, no port). The outputs below therefore prove what the SDK and your code do with a given answer, not how well a real model uses memory. In the listings model() is the function that returns your model: in the test it returns the fake one. Every listing that was run was run twice, with identical output; the listings of a single section are pieces of the same file. The stores (memory, archive) are in-RAM stand-ins, declared as such; the SQL block and the deletion function are schemas to adapt, not run. The sessions API is among the youngest parts of the SDK: pin the version and check the names.

The analogy: the colleague with anterograde amnesia

You have a brilliant colleague who, every time you leave the room, forgets everything. The only way to work with them is to hand over, at every entrance, the complete minutes of what you have said to each other. At the third entrance the minutes are three pages long; at the thirtieth, thirty - and they read everything, every time, before answering you. At some point the minutes no longer fit on the desk: you have to choose whether to bring only the last pages, or a summary, or an index with the possibility of looking up details in the archive. These three choices are exactly the three memory strategies we see below, and in practice the variants are combinations of them.

The minimal mechanism: the reused history

The simplest form of memory needs no libraries: you take the history from the previous run and use it as the input of the next one.

import asyncio
from agents import Agent, Runner
from lessons import model

agent = Agent(name="Assistant", instructions="Answer in English, briefly.", model=model())

async def main():
    result = await Runner.run(agent, "My name is Federico and I work on an Angular portfolio.")
    print(result.final_output)

    history = result.to_input_list()              # the whole conversation so far
    history.append({"role": "user", "content": "Which framework am I working on?"})

    result2 = await Runner.run(agent, history)    # input = a list, not a string
    print(result2.final_output)
    print(len(history), "items sent to the second turn")

asyncio.run(main())

# Output:
# Nice to meet you, Federico.
# Angular.
# 3 items sent to the second turn

The key point is that Runner.run accepts both a string and a list of items. Without the second case there would be no memory. The output shows what you are really doing: on the second turn you send three items (the question, the answer, the new question), so you pay for the first turn too. The items of to_input_list() do not always have the shape {"role": ..., "content": ...}: tools produce items of type function_call and function_call_output, and it is worth knowing this before writing code that cuts them (you see it in the section on truncation).

The calculation, with numbers

Suppose turns of similar size: 500 tokens between question and answer (a figure chosen for the example, not measured). At turn n the prompt contains the sum of all the previous turns, and the calculation takes a few lines.

TOKENS_PER_TURN = 500
TURNS = 20

sent_per_turn = [TOKENS_PER_TURN * n for n in range(1, TURNS + 1)]
print("turn 1:", sent_per_turn[0], "- turn 2:", sent_per_turn[1],
      "- turn 20:", sent_per_turn[-1])
print("total sent:", sum(sent_per_turn))
print("if memory were linear:", TOKENS_PER_TURN * TURNS)
print("ratio:", round(sum(sent_per_turn) / (TOKENS_PER_TURN * TURNS), 2))

# Output:
# turn 1: 500 - turn 2: 1000 - turn 20: 10000
# total sent: 105000
# if memory were linear: 10000
# ratio: 10.5

If memory were "linear" you would pay 20 × 500 = 10,000 tokens. You pay 10.5 times as much (the ratio is (n+1)/2), and the factor gets worse with the length of the conversation: the total grows like n². This is why a long chat becomes expensive long before it exhausts the context window, and why every token you put in a system prompt you pay n times, not once.

What follows from quadratic growth

The calculation of the previous section - 105,000 tokens for twenty turns instead of 10,000 - is not an arithmetic curiosity: it is the constraint that determines three design decisions that would otherwise seem arbitrary. It is worth deriving them, because they are the same three that come back in every lesson on costs.

First consequence: every token in the system prompt is paid once per remaining turn. A 200-token paragraph at the top of the instructions costs 200 tokens on the first turn and 200 on each of the following ones: in a twenty-turn conversation that is 4,000 tokens, not 200. This completely reorders the optimisation priorities. Shortening an instruction by one line that repeats is worth twenty times more than shortening an answer that is said once. It also explains why the recommendation "put the rare cases in the tool description, not in the system prompt" is not stylistic: the tool description is also paid for at every round, but at least you pay it only if that tool is in the catalogue of that agent.

Second consequence: the result of a tool is the most underrated cost item. We already met this in the lesson on tools, and here you see why: a 1,500-token result produced at turn three is sent again at turns 4, 5, 6 and the following ones. It is not a local cost, it is a cost that repeats. Hence the rule - a tool is an information reducer - with its numerical justification.

Third consequence: the break-even point between "keep everything" and "summarise" can be calculated. It is not a matter of taste. Summarising costs an extra call (the summariser) and reduces the size of the history from that point on. It pays off when the cost avoided in the following turns exceeds the cost of the compression.

def worth_summarising(history_tokens: int, expected_remaining_turns: int,
                      summary_tokens: int = 300,
                      summary_call_cost: int | None = None) -> bool:
    """Rough comparison in tokens: is compressing now worth it?"""
    # cost of doing nothing: the whole history, for every remaining turn
    without = history_tokens * expected_remaining_turns
    # cost of compressing: one read of the history + the summary repeated
    compression_cost = summary_call_cost or history_tokens
    with_ = compression_cost + summary_tokens * expected_remaining_turns
    return with_ < without

print(worth_summarising(6000, 8))     # 6000 + 2400 against 48000
print(worth_summarising(6000, 1))     # 6000 + 300 against 6000
print(worth_summarising(6000, 2))     # 6000 + 600 against 12000

# Output:
# True
# False
# True

The three lines of output are three scenarios with the same 6000-token history: with eight remaining turns compressing pays off (8,400 against 48,000), with two it still does (6,600 against 12,000), with one it does not (6,300 against 6,000). The parameter that is missing from this function, and that is the hardest to estimate, is expected_remaining_turns. In practice you use an empirical estimate from the real distribution of conversations (the example "90% within six turns" is hypothetical): if almost all of them end within six turns, summarising at the fifth is almost always a waste. It is a number you get for free in the logs, and that most projects never look at.

A note so as not to take the formula too seriously: it compares token to token, whereas in reality input and output tokens have different prices, and a call to the summariser also has a latency cost that falls exactly in the turn in which the user is waiting. It serves as an order of magnitude to understand when the question is relevant, not as an automatic decision-maker.

When there is an error in the history: consolidation

There is an effect of memory that is not about cost and that is more insidious: a wrong answer, once it has entered the history, influences all the following turns. The model rereads it at every round as part of the conversation: it is in the context, written with the "assistant" role, and nothing distinguishes it from a right answer. It is a plausible risk; in these tests I did not measure how often a real model falls for it, and the listing below shows only the mechanics.

The typical example: at turn two the agent states that order A-7781 was shipped on October 3. It is false, but nobody corrects it. At turn six the user asks how much time has passed since the shipment, and an agent that trusts the history calculates the days from October 3: perfect internal consistency, wrong result. The error has consolidated and generated a second one, which looks like the product of a correct calculation.

There are three defences, in an order that is my design judgment and not a measurement: first you correct, then you prevent, finally you cut.

Correct explicitly, do not hope. If your code notices that a statement was wrong (for example because an output guardrail compared an order number with the database), ignoring it is not enough: a message that explicitly contradicts it must be inserted in the history. Hoping that the last message prevails is not a strategy, because the model sees both. The output shows the shape of the remedy: the history goes from three to four messages, the falsehood stays (you do not delete it, because the rest of the conversation refers to it) and right after it comes the correction, which says which statement to ignore.

def contradict(history: list, wrong_claim: str, truth: str) -> list:
    """Adds an explicit correction instead of counting on the last message."""
    return [*history, {"role": "user", "content":
        f"CORRECTION: what was said earlier about «{wrong_claim}» is wrong. "
        f"The correct fact is: {truth}. "
        f"Use ONLY the correct fact from here on and ignore the previous claim."}]

history = [
    {"role": "user", "content": "Has order A-7781 left?"},
    {"role": "assistant", "content": "Yes, order A-7781 was shipped on October 3."},   # false
    {"role": "user", "content": "How many days have passed since the shipment?"},
]
new = contradict(history, "the shipment of A-7781", "A-7781 has not left yet")
print(len(history), "->", len(new), "messages")
for m in new:
    print(m["role"].ljust(9), m["content"][:70])

# Output:
# 3 -> 4 messages
# user      Has order A-7781 left?
# assistant Yes, order A-7781 was shipped on October 3.
# user      How many days have passed since the shipment?
# user      CORRECTION: what was said earlier about «the shipment of A-7781» is wr

Do not let unverified statements in. It is the best defence and it is designed upstream: if the agent must never cite a number it has not read from a tool, the way to enforce it is not an instruction ("do not make up numbers") but structure - an output type with the quote field, as in the lesson on structured output, and a check that compares it with the source.

Cut the conversation. In cases where the history has been polluted beyond recovery, starting again with a summary written by your code on the basis of verified facts - not from the history - is cheaper than trying to correct. It is brutal and sometimes it is the right thing.

It is worth noting the connection with the lesson on the real traps of agents, where there is an injection lab: consolidation is the mechanism that makes indirect injection so effective. A hostile instruction that entered the context through the output of a tool does not apply only to the turn in which it arrives: it stays there, and is sent back to the model at every following turn. Memory, which is the property that makes an assistant useful, is also the channel that amplifies hostile content.

Sessions: memory delegated to the SDK

Copying the history by hand works in a script and becomes fragile in a server. The SDK offers the concept of a session: an object that loads the history before the run and saves it afterwards, identified by a key. The file conversations.db is a local SQLite database; in the test I created it in a temporary folder.

import asyncio
from agents import Agent, Runner, SQLiteSession
from lessons import model

agent = Agent(name="Assistant", instructions="Answer in English, briefly.", model=model())

async def turn(conversation_id: str, text: str) -> str:
    session = SQLiteSession(conversation_id, "conversations.db")
    result = await Runner.run(agent, text, session=session)
    return result.final_output

async def main():
    # two independent conversations, in the same process
    print(await turn("user-42", "My name is Federico."))
    print(await turn("user-99", "My name is Giulia."))
    print(await turn("user-42", "What is my name?"))
    for key in ("user-42", "user-99"):
        items = await SQLiteSession(key, "conversations.db").get_items()
        print(key, len(items), "items saved")

asyncio.run(main())

# Output:
# Nice to meet you, Federico.
# Nice to meet you, Giulia.
# Federico.
# user-42 4 items saved
# user-99 2 items saved

The output says two things: the third answer ("Federico") arrives after a conversation interleaved with another user, and the two sessions stay separate (4 items for user-42, two questions and two answers; 2 for user-99). Then three things to know before using it in production. First: the session key is your isolation boundary - if you get it wrong, two users read each other's conversation, and that is a confidentiality incident, not a convenience bug. Second: SQLite is fine for one machine, not for several processes writing concurrently; on a real service you need a session backed by your own database. Third: by default the session limits nothing (SessionSettings(limit=None), from the documentation): it grows until it saturates the context window, and what happens then depends on the server, which may reject the request or truncate; on a real Ollama I did not verify it.

The limit exists, and it is an SDK setting. There is a trap I found by trying it: the limit counts items, not turns, and a turn with tools occupies quite a few.

import asyncio
from agents import Agent, Runner, RunConfig, SessionSettings, SQLiteSession
from lessons import model, REQUESTS, read_stock, read_incoming_orders

agent = Agent(name="Warehouse", instructions="Use the tools.", model=model(),
              tools=[read_stock, read_incoming_orders])

def kind(m: dict) -> str:
    return m.get("type") or m.get("role")

async def main():
    session = SQLiteSession("user-42", "conversations.db")
    await Runner.run(agent, "Should SKU-11 be reordered?", session=session)
    print("saved   :", [kind(m) for m in await session.get_items()])
    print("limit=4 :", [kind(m) for m in await session.get_items(limit=4)])
    REQUESTS.clear()
    await Runner.run(agent, "And SKU-42?", session=session,
                     run_config=RunConfig(session_settings=SessionSettings(limit=4)))
    sent = REQUESTS[-1]["messages"]
    print("sent    :", [m["role"] for m in sent])

asyncio.run(main())

# Output:
# saved   : ['user', 'function_call', 'function_call_output', 'function_call', 'function_call_output', 'message']
# limit=4 : ['function_call_output', 'function_call', 'function_call_output', 'message']
# sent    : ['system', 'assistant', 'tool', 'assistant', 'user']

The output reads line by line. The session saves six items for a single turn with two tools. From the request actually sent you can work out what the model receives: to read it I use the log of the fake transport (REQUESTS), which you do not have with a real server. Asking for the last four, directly from the session, returns a list that starts with a tool result without its request. In the request actually sent, instead, the roles are system, assistant, tool, assistant, user: the SDK discarded the orphan result. It is a behaviour of this version with this type of model, which I observed and which the documentation does not promise; if you manage the list yourself (with to_input_list()), nobody cleans anything up for you, as you see in the section on truncation.

Estimating tokens without depending on a tokenizer

To decide when to truncate or summarise you need a number, and you need it before making the call. The exact count depends on the tokenizer of the specific model, which on a local installation you may not have at hand. An estimate based on characters is always available; the characters-per-token ratio I use (3.2) is a cautious assumption, not a measurement: pick one lower than the true one to stay on the safe side, and check it with your model's tokenizer when you can. The function works on the text of any item, nested or not, because the SDK's items do not always have a content field.

import json

# Cautious assumption, NOT measured: your model's tokenizer may give another ratio.
CHARS_PER_TOKEN = 3.2
MESSAGE_OVERHEAD = 4        # role and delimiters, estimated

def _chars(x) -> int:
    """Characters of text in a message, in any format: nested strings included."""
    if isinstance(x, str):
        return len(x)
    if isinstance(x, dict):
        return sum(_chars(v) for v in x.values())
    if isinstance(x, list):
        return sum(_chars(v) for v in x)
    return 0

def estimate_tokens(messages: list[dict]) -> int:
    return int(sum(_chars(m) / CHARS_PER_TOKEN + MESSAGE_OVERHEAD for m in messages))

def history_budget(model_window: int, reserved_for_output: int = 1500,
                   margin: float = 0.15) -> int:
    """How many tokens the history may occupy, leaving room for the answer."""
    return int((model_window - reserved_for_output) * (1 - margin))

classic_chat = [{"role": "user", "content": "x" * 320}]
sdk_format = [{"type": "function_call", "name": "read_stock", "call_id": "c1",
               "arguments": '{"sku": "SKU-11"}'},
              {"type": "function_call_output", "call_id": "c1",
               "output": "{'sku': 'SKU-11', 'stock': 4}"}]
print(estimate_tokens(classic_chat))     # (320 + 4 characters) / 3.2 + 4: the role counts too
print(estimate_tokens(sdk_format))
print(history_budget(8192))              # (8192 - 1500) * 0.85

# Output:
# 105
# 37
# 5688

The three outputs read like this. The first is a classic chat of 320 characters: 105 estimated tokens, because even the word "user" of the role counts as text (a conservative choice) and 4 tokens of overhead per message are added. The second is a request/result pair in the SDK's format: 37 tokens, which the naive version (the one that looks only at content and tool_calls) would have estimated at 8 tokens, that is the overhead alone. The third is the budget for a window of 8192 tokens: 5688, after reserving 1500 tokens for the answer and a 15% margin. Forgetting the room for the answer is the classic way to get a full-context error right after having paid for the whole prompt.

The three strategies, in a drawing

When the history no longer fits in the window, there are three options and they are told apart by what they throw away. Keeping them in mind as an image helps you choose quickly.

The three strategies when it no longer fits

1. Sliding-window truncation. You keep the system prompt and the last k turns. Cheap, predictable, and it loses the past brutally: the user will say "as I told you earlier" and the agent will not know what they are talking about.

import asyncio
from agents import Agent, Runner
from lessons import model, read_stock, read_incoming_orders

def is_tool_result(m: dict) -> bool:
    # in the format the SDK puts in to_input_list() the result is an item
    # of type function_call_output; in the classic chat format it is role == "tool"
    return m.get("type") == "function_call_output" or m.get("role") == "tool"

def truncate(history: list, last: int = 12) -> list:
    system = [m for m in history if m.get("role") == "system"]
    rest = [m for m in history if m.get("role") != "system"]
    tail = rest[-last:]
    # do NOT start with a result whose request was cut off
    while tail and is_tool_result(tail[0]):
        tail = tail[1:]
    return system + tail

def kind(m: dict) -> str:
    return m.get("type") or m.get("role")

async def main():
    agent = Agent(name="Warehouse", instructions="Use the tools.", model=model(),
                  tools=[read_stock, read_incoming_orders])
    history = (await Runner.run(agent, "Should SKU-11 be reordered?")).to_input_list()
    print("whole          :", [kind(m) for m in history])
    print("last 4         :", [kind(m) for m in history[-4:]])
    print("after truncate :", [kind(m) for m in truncate(history, 4)])

asyncio.run(main())

# Output:
# whole          : ['user', 'function_call', 'function_call_output', 'function_call', 'function_call_output', 'message']
# last 4         : ['function_call_output', 'function_call', 'function_call_output', 'message']
# after truncate : ['function_call', 'function_call_output', 'message']

The output shows the flaw. The complete history has six items; the last four start with a function_call_output whose request was cut off; after truncate the orphan is gone and three items remain. In the version I originally wrote the check looked at role == "tool", which in the format of to_input_list() does not exist: the while would never have removed anything. I also verified that, when you pass the SDK a list that starts with an orphan result, the request goes out as it is (roles system, tool, user): whether a real endpoint rejects it, and which one, I did not test, but the cautious rule is the same. Request/result pairs are atomic.

2. Progressive summary. When the history exceeds a threshold, you ask a model to compress the old turns into a summary and replace them with it.

THRESHOLD = 6000       # estimated tokens

async def compress(history: list, summariser: Agent, threshold: int = THRESHOLD) -> list:
    if estimate_tokens(history) < threshold:
        return history
    cut = len(history) - 6                       # keep about the last 6 items
    while cut > 0 and is_tool_result(history[cut]):
        cut -= 1                                 # the request/result pair stays whole
    head, tail = history[:cut], history[cut:]
    result = await Runner.run(summariser, [
        *head,
        {"role": "user", "content":
            "Summarise the conversation above in at most 200 words. "
            "Keep: decisions made, stated constraints, identifiers, exact numbers, "
            "things still to do. Discard greetings and rephrasings."}
    ])
    return [{"role": "system", "content": f"Summary of the previous conversation:\n{result.final_output}"},
            *tail]

To test it I built a conversation of 30 items and a fake summariser, which answers with a text written by me.

summariser = Agent(name="Summariser", instructions="Compress without losing the identifiers.",
                   model=model())

async def main():
    history = [{"role": "user" if i % 2 == 0 else "assistant",
                "content": f"turn {i}: " + "words " * 120} for i in range(30)]
    new = await compress(history, summariser)
    print(len(history), "items,", estimate_tokens(history), "estimated tokens")
    print(len(new), "items,", estimate_tokens(new), "estimated tokens")
    print(new[0]["content"][:60])
    print(new[1]["content"][:8])

asyncio.run(main())

# Output:
# 7 items, 1449 estimated tokens
# Summary of the previous conversation:
# Summary: the user is F
# turn 24:

The output shows the mechanics: 30 items and 7,012 estimated tokens become 7 items and 1,449 (the summary plus the last six, which here are worth about 230 tokens each). The text of the summary is my script, not a proof of how well a model compresses. The cut moves back if it falls on a tool result, for the same reason as before. The summariser's prompt is the part that matters: "keep identifiers and exact numbers" is what avoids losing the order code that will be needed six turns from now. The structural flaw remains: the summary is lossy and the error accumulates, because summary n+1 is based on summary n. After many compressions the conversation tells an approximate version of itself.

3. Retrievable external memory. In the context you keep only the recent window; everything else goes into an archive (text or vector search) and comes back only when needed, through a tool.

import asyncio
from agents import Agent, Runner, function_tool
from lessons import model

class Archive:
    """Stand-in for your archive: search by words in common, no vector index."""
    def __init__(self, messages: dict[str, list[str]]):
        self.messages = messages
    async def search(self, session_id: str, question: str, k: int) -> list[str]:
        words = set(question.lower().split())
        score = lambda t: len(words & set(t.lower().replace(".", "").split()))
        candidates = sorted(self.messages.get(session_id, []), key=score, reverse=True)
        return [t for t in candidates[:k] if score(t) > 0]

archive = Archive({"s1": ["My order number is A-7781.", "I prefer fast shipping.",
                          "Good morning!"]})

def current_session() -> str:
    return "s1"                       # in your code: the id of the conversation in progress

@function_tool
async def search_history(question: str, max_results: int = 3) -> dict:
    """Search the past messages of THIS conversation, beyond the recent window.

    Use this tool when the user refers to something said 'earlier'
    and you cannot find it in the messages in front of you.
    """
    passages = await archive.search(current_session(), question, k=max_results)
    return {"fragments": passages, "note": "extracts from past messages, not paraphrased"}

agent = Agent(name="Assistant", instructions="Search the history if needed.",
              tools=[search_history], model=model())

async def main():
    result = await Runner.run(agent, "What was my order number, the one I gave earlier?")
    for item in result.new_items:
        if item.type == "tool_call_output_item":
            print(item.output)

asyncio.run(main())

# Output:
# {'fragments': ['My order number is A-7781.'], 'note': 'extracts from past messages, not paraphrased'}

The archive in the listing is an in-RAM stand-in that compares words: the output proves that the tool returns the right fragment for that question, not that a real vector search would find it. It is the best strategy on long conversations and the only one that really scales, but it introduces one more component (the archive and its index) and a new way of going wrong: retrieval may return the wrong fragment and the model will believe it. The architectural side - indexes, knowledge graphs, long-term memory across different sessions - is covered in depth in the dedicated series, memory systems for agents: context and knowledge graphs; here I stop at the level of the single conversation.

A persistent session is an archive of personal data

There is an aspect of memory that is not technical and that must be decided before writing the code, because afterwards it is a job of removal instead of design: the moment you save conversations to disk, you have created an archive of personal data. Not because you meant to, but because users write things in chats that they would not write in a form: addresses, order numbers, health problems, family situations, complaints about colleagues.

Three decisions to take explicitly, with their technical consequence.

What you save. The default choice "everything" is convenient and almost never necessary. Often at the next turn you need the last messages and a small structured state (the order number under discussion, the language, the decision already taken), not the full transcript. Saving the structured state instead of the transcript reduces at once the cost, the risk and the exposure surface. It is not always possible; when it is, it is the best choice on every axis.

For how long. An expiry must be decided and implemented, not postponed. The implementation is trivial and the fact that it is trivial is the reason not to put it off.

-- example with PostgreSQL: expiry declared on the row (schema to adapt, not executed)
CREATE TABLE conversations (
    id          text PRIMARY KEY,
    user_id     text NOT NULL,
    messages    jsonb NOT NULL,
    created_at  timestamptz NOT NULL DEFAULT now(),
    expires_at  timestamptz NOT NULL DEFAULT now() + interval '30 days'
);
CREATE INDEX ON conversations (expires_at);
CREATE INDEX ON conversations (user_id);

-- periodic deletion: one line, from a cron job
DELETE FROM conversations WHERE expires_at < now();

How to delete on request. There must be a function that, given a user, removes everything: sessions, durable memory, and - this is the point that gets forgotten - also the traces and the logs. A conversation deleted from the database and still present in a tracing system is not deleted. It is worth writing that function on the first day, because discovering it is missing when it is needed is a bad day.

# schema to adapt, not executed: db, tracing and vector_index are your components
async def delete_everything_of(user_id: str) -> dict:
    """Removes every trace of a user. To be called on a deletion request."""
    sessions = await db.execute("DELETE FROM conversations WHERE user_id = $1", user_id)
    facts    = await db.execute("DELETE FROM durable_memory WHERE user_id = $1", user_id)
    traces   = await tracing.delete_for_user(user_id)   # do NOT forget this one
    archive  = await vector_index.delete_for_user(user_id)   # nor this one
    return {"sessions": sessions, "facts": facts,
            "traces": traces, "indexed_fragments": archive}

This last point has a consequence that touches the architecture and is worth anticipating: the strategy of retrievable external memory - the one that indexes past messages in a search archive - multiplies the places where a person's data exists. It is the best strategy for quality and the worst for the life cycle of the data, because a vector index must be cleaned with the same diligence as the database. It is not a reason to avoid it, it is a reason to write the deletion function together with the indexing and not six months later.

An observation that closes the circle with the rest of the module: on this ground a model run locally changes the nature of the problem. If the model runs on your machine, the contents of the conversations do not leave it, so the perimeter to protect is only one and you control it. All the work on retention and deletion remains - no choice of provider solves that - but the hardest part to govern disappears, that is the data you have already sent to someone else.

Session memory and durable memory are two different things

It is worth separating them in the code too, because they have different retention policies:

• Session. Lives as long as the conversation, contains everything, can be thrown away. Lasts minutes or hours.
• Durable. Stable facts about the user ("prefers short answers", "works on Angular", "time zone Europe/Rome"). A few lines, written with intent, reread at the start of every new conversation.

Writing to durable memory is a decision, not a side effect: if everything ends up in it, it becomes a second context that you always pay for and that contains noise. Two lines at the top of the system prompt beat thirty lines of history.

Durable memory written by the agent itself

So far we have described durable memory as something you write. You can also let the agent decide what is worth remembering, and it is useful - provided you treat it as a constrained operation, not as a free diary.

import asyncio
from dataclasses import dataclass
from typing import Literal
from agents import Agent, Runner, function_tool, RunContextWrapper
from lessons import model

@dataclass
class Fact:
    category: str
    content: str

@dataclass
class User:
    user_id: str

class InMemoryStore:
    """Stand-in for your store: a dictionary, no database."""
    def __init__(self, preloaded: int = 0):
        self.per_user: dict[str, list[Fact]] = {
            "u1": [Fact("stable_fact", f"fact {i}") for i in range(preloaded)]}
    async def read(self, user_id: str) -> list[Fact]:
        return list(self.per_user.get(user_id, []))
    async def add(self, user_id: str, category: str, content: str) -> None:
        self.per_user.setdefault(user_id, []).append(Fact(category, content))

memory = InMemoryStore()

@function_tool
async def remember(context: RunContextWrapper[User],
                   category: Literal["preference", "constraint", "stable_fact"],
                   content: str) -> dict:
    """Save ONE stable fact about the user, useful in future conversations too.

    Use this tool only for things that stay true tomorrow: style preferences,
    technical constraints, personal details stated by the user.
    Do NOT use it for: the state of the conversation in progress, numbers that change,
    things said once by chance.
    """
    if len(content) > 180:
        return {"ok": False, "error": "too_long",
                "how_to_fix": "Rewrite it in fewer than 180 characters, one fact only."}
    facts = await memory.read(context.context.user_id)
    if len(facts) >= 20:                       # hard ceiling
        return {"ok": False, "error": "memory_full",
                "how_to_fix": "Delete an outdated fact (a forget tool is needed), then retry."}
    await memory.add(context.context.user_id, category, content)
    return {"ok": True, "stored_facts": len(facts) + 1}

agent = Agent(name="Assistant", instructions="Remember the stable facts.", tools=[remember],
              model=model())

async def main():
    result = await Runner.run(agent, "I prefer short answers.", context=User("u1"))
    for item in result.new_items:
        if item.type == "tool_call_output_item":
            print(item.output)

asyncio.run(main())

# Output:
# {'ok': True, 'stored_facts': 1}
# {'ok': False, 'error': 'too_long', 'how_to_fix': 'Rewrite it in fewer than 180 characters, one fact only.'}

In the test the script calls the tool twice: the first time with a valid fact (answer ok: True), the second with 181 characters (answer too_long, with the remedy written for the model). Repeated with a memory that starts from 20 facts (InMemoryStore(preloaded=20)), the same call answers:

# Output:
# {'ok': False, 'error': 'memory_full', 'how_to_fix': 'Delete an outdated fact (a forget tool is needed), then retry.'}

The ceiling of twenty facts is not excess caution: it is the difference between a useful memory and a second context that you pay for at every turn of every future conversation. Without a limit, after a month the durable memory contains "the user said hello", "the user asked how you are" and everything the model judged interesting in a moment of enthusiasm.

And the use, at the start of each new conversation (the Fact class and the memory are those of the previous listing, here reduced to a stand-in):

import asyncio
from dataclasses import dataclass

INSTRUCTIONS = "Answer in English, briefly."

@dataclass
class Fact:
    category: str
    content: str

class Memory:
    async def read(self, user_id: str) -> list[Fact]:
        return [Fact("preference", "short answers"), Fact("stable_fact", "works on Angular")]
memory = Memory()

async def open_conversation(user_id: str) -> list[dict]:
    facts = await memory.read(user_id)
    if not facts:
        return [{"role": "system", "content": INSTRUCTIONS}]
    lines = "\n".join(f"- [{f.category}] {f.content}" for f in facts)
    return [{"role": "system",
             "content": f"{INSTRUCTIONS}\n\nThings you already know about this user:\n{lines}\n"
                        "Use them without announcing them. If one contradicts what the user "
                        "says now, what they say now wins."}]

async def main():
    history = await open_conversation("u1")
    print(history[0]["content"])

asyncio.run(main())

# Output:
# Answer in English, briefly.
#
# Things you already know about this user:
# - [preference] short answers
# - [stable_fact] works on Angular
# Use them without announcing them. If one contradicts what the user says now, what they say now wins.

The last line of the instructions is the one that avoids the most annoying effect of these systems: the agent that insists on a three-month-old preference while the user is telling it the opposite. Precedence to the present has to be written explicitly: there is no reason to leave to the model the guess about which of the two sources counts more. In the test I only verified that the text is composed as intended.

Why "more context" is not "more memory"

There is a line of reasoning that seems obvious and is false: if context windows grow, the memory problem solves itself - just send everything. It is worth taking it apart, because it misdirects design choices and because the way in which it is false is instructive.

The first reason is the arithmetic one we have already done: a larger window does not change the fact that you pay for the history at every turn. A window ten times larger allows conversations ten times longer, and the cost of those conversations grows a hundred times. The ceiling moves, the curve does not.

The second reason is the interesting one: the ability to use information well does not grow with the window. The work of Liu and colleagues (Lost in the Middle: How Language Models Use Long Contexts, arXiv:2307.03172) measures this directly on two tasks (questions over several documents and retrieval of key-value pairs) in which the needed information is placed in different positions of a long context, and finds that performance is often highest when it is at the beginning or at the end and drops when it is in the middle, a U-shaped trend: information at the beginning and at the end is used better than information in the centre. The practical consequence is concrete and applies at once.

• Instructions go at the beginning (where they already are, by construction of the message format).
• The current request and the data just read go at the end, as close as possible to the point where the model generates.
• What is in the middle is the part the model will use worst. If a critical piece of information is buried in the middle of four thousand tokens of tool results, it must be repeated at the end, not left there hoping.

The list translates into a rule for assembling the context, and the third line is a pattern worth naming, because it solves a class of problems: re-assertion. When a run has accumulated a lot of context and you must be sure a constraint is respected, the effective way is not to make it more emphatic in the system prompt (which is far away) but to repeat it in a message immediately before the decision.

def with_reassertion(history: list, constraints: list[str]) -> list:
    """Repeats the critical constraints at the END of the context, where they are used best."""
    if not constraints:
        return history
    lines = "\n".join(f"- {c}" for c in constraints)
    return [*history, {"role": "user", "content":
        f"Before answering, remember the constraints that always apply:\n{lines}"}]

history = [{"role": "system", "content": INSTRUCTIONS}]
history = with_reassertion(history, [
    "Do not exceed 1000 EUR without approval: use request_approval.",
    "Always cite the order number read from the tools, never from memory.",
])
print(len(history), "messages; the last one has the role", history[-1]["role"])
print(history[-1]["content"])

# Output:
# 2 messages; the last one has the role user
# Before answering, remember the constraints that always apply:
# - Do not exceed 1000 EUR without approval: use request_approval.
# - Always cite the order number read from the tools, never from memory.

It costs a few dozen tokens. The output proves that the re-assertion message ends up last; whether it changes the behaviour of a real model in long conversations I have not measured, and I present it as a reasonable application of a property documented by the work of Liu et al., not as a result of mine.

The third reason why "send everything" is not a strategy concerns noise. A long history contains, together with the useful information, all the attempts that went badly: the tool called with the wrong argument, the error returned, the user's question rephrased three times. All this material is consistent with the task and irrelevant for the decision, and its presence makes it harder to tell the signal. A well-written summary - which keeps decisions, identifiers and constraints, and throws away the failed attempts - can produce better answers than the full history, not just cheaper ones. It is the case in which the two optimisations, cost and quality, point in the same direction: it rarely happens, and when it does it is worth noticing.

Trade-offs

• Fidelity versus cost. Complete history: remembers everything, costs n². Summary: linear cost, loses details. Retrieval: low and precise cost, but depends on the quality of the search.
• Latency. Summarising costs one extra call exactly in the turn in which the user is waiting. It is mitigated by compressing before the critical threshold, in the background.
• Privacy and retention. A persistent session is an archive of personal data: it requires an expiry, deletion on request and an explicit decision about what is saved.
• Quality on long contexts. More context is not always better: the work of Liu et al. (Lost in the Middle: How Language Models Use Long Contexts, arXiv:2307.03172) documents that information placed in the middle of a long context is used worse than information at the beginning or at the end. Practical consequence: put what matters at the beginning or at the bottom, not in the middle.

In practice the four items read together: if the priority is to lose nothing you pay in cost or in latency, if the priority is cost you pay in lost details, and privacy is a constraint that applies whatever strategy you choose.

The traps people really fall into

1. Breaking a tool request/result pair by truncating. Validation error from the endpoint, cryptic message, half an hour lost. Truncate by complete turns.
2. Wrong or reused session key. Using a predictable identifier not tied to the authenticated user means being able to read someone else's conversation by changing a number in the URL.
3. Memory that grows without a ceiling. No limit in the session = the application works for two weeks and then starts rejecting the requests of the most assiduous users.
4. Summarising too early. Compressing after three turns saves nothing and loses details that were needed. The threshold must be calculated on the real window of the model, not set by intuition.
5. Putting volatile data in the system prompt. You pay for it at every turn and it goes stale: today's date, the cart, the stock belong in a tool.
6. Confusing memory and truth. If there is a wrong answer from the model in the history, it stays in the context and influences all the following turns: the error consolidates. It is worth correcting explicitly instead of hoping that the last message prevails.

The order of the six traps is not a scale: they are independent ways of going wrong. The first and the second are found at the first serious test, the third and the fourth only after weeks of use, the fifth and the sixth are the ones the code never signals by itself.

What you can do now

You can calculate how much a growing conversation costs and decide with a comparison when to compress it; you can choose between truncating, summarising and retrieving, and write the truncation without breaking a request/result pair in the format the SDK really produces; you can use sessions while limiting their size and design expiry and deletion of the data; you can assemble the context so that the critical instructions sit where they are read best. What stays out is what I could not test here: how a real model, on Ollama, behaves with a long or summarised history. In the next lesson the conversation splits in two: what happens when one agent hands control over to another.

Sources

• OpenAI Agents SDK: Sessions - SQLiteSession, SessionSettings(limit=N) (default None: the whole history), session_input_callback, consulted on 2026-11-12; the behaviour with orphan tool results is my own observation with 0.24.0
• Liu et al., Lost in the Middle: How Language Models Use Long Contexts (arXiv:2307.03172) - performance often highest with the information at the beginning or at the end of the context and dropping in the middle, abstract read on 2026-11-12

Run twice with identical output (Python 3.14.4, openai-agents 0.24.0) against a fake model with a simulated HTTP transport: reused history, token count, convenience comparison, correction, sessions and limit, token estimate, truncation, compression, search in the history, the remember tool, composition of the opening and re-assertion. Not run: the SQL block, the deletion function, any call to a real model (not even Ollama). The examples 90%, 500 tokens per turn and A-7781 are hypothetical.