07 - Memory Systems: Context, History and Knowledge Graphs

The problem: an agent that starts from zero every time

A language model remembers nothing between one call and the next: what it “knows” about your conversation is what you send back in the prompt. An agent's memory is therefore a problem of choice, not of storage: what to put back in the prompt, where to take it from, when to stop carrying it around. This article gives three answers to three different questions.

For the session in progress: a window of recent messages and, if needed, a summary of the old ones. For documents and notes: similarity search over chunks of text. For relationships and for facts that change: a graph, with a validity in time. At the end a fourth listing puts the results together inside a token budget.

The listings run with only the Python 3.14.4 standard library (SQLite included) and no model: the token counts are a declared estimate of one word per token, the summarizer and the “embedding” are rule-based toys, and where a result depends on a real model I say so. If you first need the picture of coordination between agents, start from Multi-Agent Orchestration: the shared state there is a log of events, here it is what the agent carries with it from one turn to the next.

What is meant by memory

The LangChain documentation distinguishes two scopes. Short-term memory is tied to a conversation (a “thread”): in LangGraph it is part of the agent's state, and it is saved to a database through a checkpointer so that the thread can be resumed; the article on LangChain and LangGraph covers it. Long-term memory keeps data about a user or the application across sessions, is shared between threads and lives in “namespaces” you choose.

As for the content, the same documentation uses the tripartition from psychology: semantic memory (facts), episodic (experiences), procedural (rules on how to carry out a task). The work that organized the field, “Cognitive Architectures for Language Agents” (CoALA), describes an agent as having modular memory components, with a structured set of actions to read and write them.

The table corrects a widespread habit: making episodic memory coincide with a knowledge graph. In the documentation cited, episodic memory is experiences, often reintroduced as examples in the prompt; the graph is a tool for representing facts and relationships, which can serve several types. Even the “sensory, short, long, episodic” classification found in many tutorials is an analogy with psychology, not a structure of the frameworks. In the rest of the article the useful question is another one: for each piece of information, how long must it live, and how do I find it again when needed.

The session: window and summary

The history of a conversation grows at every turn and every message you send back costs tokens and latency. The simplest solution is the sliding window: you keep the most recent messages that fit in a limit and discard the rest. It is predictable and needs no further model calls, but it has a flaw you see at once in the listing: the instructions given at the beginning are the first to disappear.

The summary compensates: the discarded messages are condensed into an entry placed in front of the window. In the listing the summarizer is a rule (it keeps the messages that contain words such as “budget” or “DECISION”), because a real summary requires a model, that is, one more call to pay for.

def tokens(text):
    """Deliberately crude estimate: one word = one token. A real count depends on the model."""
    return len(text.split())

HISTORY = [
    ("user", "The maximum project budget is 500 euros, never exceed it."),
    ("agent", "Understood. I'll start by comparing the two libraries."),
    ("user", "Please, no external dependencies at all."),
    ("agent", "I propose the standard library."),
    ("user", "DECISION: we use SQLite for local data."),
    ("agent", "Recorded. I'm preparing the table schema."),
    ("user", "Add an index on the date column."),
    ("agent", "Done. The date column is indexed."),
]

def window(history, limit):
    """Keeps the most recent messages as long as they fit in the token limit."""
    kept, used = [], 0
    for role, text in reversed(history):
        if used + tokens(text) > limit:
            break
        kept.append((role, text))
        used += tokens(text)
    return list(reversed(kept))

def summarize(messages):
    """Fake rule-based summarizer: keeps the messages with constraints or decisions."""
    keys = ("budget", "DECISION", "no external dependencies")
    return " | ".join(t for _, t in messages if any(k in t for k in keys))

def with_summary(history, limit):
    recent = window(history, limit)
    old = history[: len(history) - len(recent)]
    return ([("summary", summarize(old))] if old else []) + recent

for name, context in (("window only", window(HISTORY, 20)),
                      ("window + summary", with_summary(HISTORY, 20))):
    total = sum(tokens(t) for _, t in context)
    remembers_budget = any("budget" in t for _, t in context)
    print(f"{name}: {len(context)} items, {total} tokens, remembers the budget: {remembers_budget}")
print("full history:", sum(tokens(t) for _, t in HISTORY), "tokens")
print(with_summary(HISTORY, 20)[0])

# Output:
# window only: 3 items, 19 tokens, remembers the budget: False
# window + summary: 4 items, 44 tokens, remembers the budget: True
# full history: 55 tokens
# ('summary', 'The maximum project budget is 500 euros, never exceed it. | Please, no external dependencies at all. | DECISION: we use SQLite for local data.')

The window alone keeps three messages for 19 tokens and has lost the budget constraint, which the user had declared first: if the agent proposes an 800-euro solution, nothing stops it. With the summary the constraint stays, at a cost of 44 tokens against the 55 of the full history.

The saving is small because the example history is short, and the summary here is a literal copy: with a real model you would also have to pay for the call that produces it and accept that it may omit a detail. That is why it is a good idea to save separately (outside the summary) the constraints that must never be lost, and to put them back at the top of every prompt.

The long term: chunks, vectors and search

When the information does not fit in a conversation (documentation, notes, code) you use retrieval: you look for the chunks of text closest to the question and put them in the prompt. The idea has an originating paper, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (Lewis et al.), which combines the memory in a model's parameters with an external memory, a vector index of Wikipedia consulted by a retriever.

An embedding model turns a text into a vector, and the closeness between vectors (usually the cosine) is the measure of similarity. The number of dimensions depends on the model: the all-MiniLM-L6-v2 model card declares 384 dimensions and says that texts beyond 256 “word pieces” are truncated by default.

That last detail decides how to split documents: a chunk longer than the model's limit is silently truncated, and the tail of the chunk does not take part in the search. The listing splits a text into chunks of twelve words with three of overlap, and uses a toy “embedding”, a word count, to show the mechanics without a model: the real model produces a dense vector and picks up synonyms, the toy one does not, and that is exactly the limit the second experiment makes visible.

import math
from collections import Counter

def chunks(text, size, overlap):
    """Splits into chunks of `size` words; each chunk repeats `overlap` words of the previous one."""
    words = text.split()
    step = size - overlap
    return [" ".join(words[i:i + size])
            for i in range(0, max(len(words) - overlap, 1), step)]

def vector(text):
    """Toy embedding: word count. A real model will give a dense vector."""
    return Counter(w.strip(".,?").lower() for w in text.split() if len(w) > 3)

def cosine(a, b):
    dot = sum(a[k] * b[k] for k in a if k in b)
    norm = math.sqrt(sum(v * v for v in a.values())) * math.sqrt(sum(v * v for v in b.values()))
    return dot / norm if norm else 0.0

DOC = ("The security certificate of the site is renewed every ninety days with the command for renewal. "
       "The database backup is run each night at eleven fifty-nine. "
       "To change the administrator password use the users panel.")
for i, c in enumerate(chunks(DOC, 12, 3)):
    print(i, c)

archive = chunks(DOC, 12, 3)
for question in ("when is the certificate renewed?", "how do I handle the renewal of the site license?"):
    scores = sorted(((cosine(vector(question), vector(c)), i) for i, c in enumerate(archive)),
                    reverse=True)
    print(f"{question!r} -> chunk {scores[0][1]}, similarity {scores[0][0]:.2f}")

# Output:
# 0 The security certificate of the site is renewed every ninety days with
# 1 ninety days with the command for renewal. The database backup is run
# 2 backup is run each night at eleven fifty-nine. To change the administrator
# 3 change the administrator password use the users panel.
# 'when is the certificate renewed?' -> chunk 0, similarity 0.41
# 'how do I handle the renewal of the site license?' -> chunk 1, similarity 0.19

The overlap is visible in chunks 0 and 1: the sentence about renewal is cut in the middle by the first cut, but chunk 1 picks it up with “ninety days with the command for renewal”, so the meaning is not lost. The first question finds the right chunk (similarity 0.41).

The second uses “license” instead of “certificate” and “renewal” instead of “renewed”: the words do not coincide, and the system chooses chunk 1, which happens to contain “renewal”, instead of chunk 0, which does answer but shares with the question only “site”; the score is low (0.19). A real embedding model should bring these terms closer, but I did not verify it here, and the moral remains: the score of a retrieval has to be looked at, and below a threshold (to be chosen on your data) it is better to answer “I don't know” than to hand the model an irrelevant chunk.

Searching among millions of vectors by comparing them all is slow, which is why vector databases use approximate indexes. The HNSW index, described by Malkov and Yashunin, builds a multi-layer graph and declares a complexity that scales logarithmically; the pgvector extension for PostgreSQL offers both exact search and the approximate HNSW and IVFFlat indexes, and its README specifies that the approximate ones trade some recall for speed, so the same data can give different results after you create the index. Before choosing a product, four decisions matter more than the brand:

• the size of the chunks, below the limit of the embedding model you use;
• the overlap, which costs space but avoids cutting a concept in two;
• the metadata (source, date, type), which let you filter and cite;
• the way a document is deleted, which must remove its vectors too.

The four items have no order of importance, but the first and the last are the ones discovered late: the first when the answers ignore the end of the documents, the last when someone asks to have their data deleted. I do not give a “right” size or overlap: they depend on the model and the documents, and are decided by comparing the quality of retrieval on questions of your own (as in Testing and Evaluation).

The graph: relationships and facts that change

Similarity search finds similar texts, it does not answer “which other bugs are in this one's project” or “what is true now”. For these questions you need a structure of facts: subject, relation, object. The listing builds it in a single SQLite table and answers three questions.

The first is a hop (a join of the table with itself). The second follows the chain of dependencies with a recursive query (WITH RECURSIVE), which SQLite supports. The third handles change: a new fact that contradicts the old one does not delete it, but closes its validity by writing the until column.

import sqlite3

db = sqlite3.connect(":memory:")
db.execute("""CREATE TABLE facts (subject TEXT, relation TEXT, object TEXT,
                                  since INTEGER, until INTEGER)""")      # until NULL = still true
def add(s, r, o, t):
    db.execute("INSERT INTO facts VALUES (?, ?, ?, ?, NULL)", (s, r, o, t))

def replace(s, r, o, t):
    """A new fact that contradicts the old one closes it, it does not delete it."""
    db.execute("UPDATE facts SET until = ? WHERE subject = ? AND relation = ? AND until IS NULL",
               (t, s, r))
    add(s, r, o, t)

for s, r, o in [("BUG-142", "in_project", "Shop"), ("BUG-198", "in_project", "Shop"),
                ("BUG-142", "fixed_by", "Marta"), ("Shop", "depends_on", "Auth"),
                ("Auth", "depends_on", "Redis"), ("Redis", "depends_on", "Network")]:
    add(s, r, o, 1)
add("Marta", "prefers", "JavaScript", 1)
replace("Marta", "prefers", "Python", 5)

# 1. One hop: other bugs in the same project as BUG-142
print(db.execute("""SELECT b.subject FROM facts a JOIN facts b
                    ON a.object = b.object AND b.relation = a.relation
                    WHERE a.subject = 'BUG-142' AND a.relation = 'in_project'
                      AND b.subject != a.subject""").fetchall())

# 2. Several hops: everything Shop depends on, with the distance
print(db.execute("""WITH RECURSIVE dep(node, distance) AS (
                      SELECT object, 1 FROM facts WHERE subject = 'Shop' AND relation = 'depends_on'
                      UNION
                      SELECT f.object, d.distance + 1 FROM facts f JOIN dep d ON f.subject = d.node
                      WHERE f.relation = 'depends_on')
                    SELECT node, distance FROM dep ORDER BY distance""").fetchall())

# 3. What is true now, and what was true before
print("now:", db.execute("""SELECT object FROM facts WHERE subject = 'Marta'
                            AND relation = 'prefers' AND until IS NULL""").fetchall())
print("history:", db.execute("""SELECT object, since, until FROM facts WHERE subject = 'Marta'
                                AND relation = 'prefers' ORDER BY since""").fetchall())

# Output:
# [('BUG-198',)]
# [('Auth', 1), ('Redis', 2), ('Network', 3)]
# now: [('Python',)]
# history: [('JavaScript', 1, 5), ('Python', 5, None)]

The first result says that in the same project as BUG-142 there is BUG-198. The second lists everything Shop depends on with the distance (Auth at 1, Redis at 2, Network at 3): a similarity search does not reconstruct a chain, because no chunk of text contains it whole. The third separates the present (Python) from the history, in which JavaScript held until time 5.

That history is useful and risky: if the facts concern a person, Regulation (EU) 2016/679 gives them in Article 17 the right to obtain the erasure of personal data concerning them, under certain conditions, so the project must also provide a path that really deletes the rows, not just closes them. Establishing whether and when that right applies to your case is not a technical question.

In a real graph, such as those queried with Neo4j's Cypher language, the same first question is written by describing the shape of the graph: nodes in round brackets, relationships in square ones with an arrow. The next listing uses the schema of the documentation (a MATCH pattern with RETURN) applied to the bugs; it is a schema to adapt, I did not run it because a Neo4j instance would be needed.

// Schema to adapt, not run: it needs a Neo4j instance with these nodes and relationships.
// The other bugs in the project where BUG-142 sits.
MATCH (:Bug {id: 'BUG-142'})-[:IN_PROJECT]->(p:Project)<-[:IN_PROJECT]-(other:Bug)
RETURN other.id AS bug, p.name AS project

The pattern reads from left to right: a Bug node with that identifier, the relationship to a Project, and then, going up the same relationship in the opposite direction, the other bugs. When the set of facts is small and the questions are few, a table like the one above is enough and does not add a service to manage; a graph database becomes interesting when the paths are long and variable, and it is a decision to make with a measurement on your data, not with a rule.

Mem0: what the source says

Mem0 is an open-source project (Apache 2.0 license, installable with pip install mem0ai according to its README) and a research paper, “Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory”. The authors describe an architecture that dynamically extracts, consolidates and retrieves salient information from ongoing conversations, and a variant with graph memory.

On the LOCOMO benchmark they declare a 26% relative improvement in the “LLM-as-a-Judge” metric over one of the comparison systems (OpenAI), a score about 2% higher for the graph variant than for the base one, 91% lower latency at the 95th percentile and over 90% lower token cost than passing the whole history in the context.

These are the authors' results on their own benchmark, which is why I attribute them: I did not reproduce them, because they would require a model, an embedding model and a store. The honest reading is that the paper supports an advantage over passing the whole context, and it is an argument in favor of selective memory; it does not say that “combining vectors and graph” is the practice to adopt everywhere, and the difference between base and graph (about 2%) is small compared to the cost of running one more component. I did not find a primary source for the idea that it is the standard of 2026, and I do not repeat it.

Assembling the context: budget, duplicates, position

Once you have candidates from several sources, you have to decide what goes into the prompt. Three rules are easy to code: a token budget, the removal of duplicates (search often returns the same fact twice) and an order. For the order there is a source:

“Lost in the Middle” (Liu et al.) shows that model performance is often highest when the relevant information is at the beginning or at the end of the context and gets worse, even in long-context models, when it sits in the middle. The listing combines two declared scores (similarity and graph, constants invented for the example), takes the candidates in score order until they fit the budget, and arranges them with the most relevant at the extremes.

def tokens(text):
    return len(text.split())            # crude estimate: one word = one token

# Candidates: (text, score from the similarity search, score from the graph).
# The scores are declared constants: what matters here is the assembly, not the search.
CANDIDATES = [
    ("The site certificate renews every ninety days.", 0.80, 0.10),
    ("Renewal runs the command provided by the certificate manager.", 0.60, 0.30),
    ("The backup starts every night.", 0.15, 0.00),
    ("The site depends on the authentication service.", 0.10, 0.70),
    ("The site certificate renews every ninety days.", 0.70, 0.00),   # duplicate
]

def assemble(candidates, budget, graph_weight=0.5):
    seen, chosen, used = set(), [], 0
    ranked = sorted(candidates, key=lambda c: (c[1] + graph_weight * c[2]), reverse=True)
    for text, vec, graph in ranked:
        if text in seen or used + tokens(text) > budget:
            continue                      # no duplicates, no overruns
        seen.add(text)
        chosen.append((len(chosen) + 1, text))     # (relevance rank, text)
        used += tokens(text)
    # the most relevant at the extremes, the least relevant in the middle
    front, back = chosen[0::2], chosen[1::2]
    return front + back[::-1], used

for budget in (40, 14):
    context, used = assemble(CANDIDATES, budget)
    print(f"budget {budget}: {used} tokens, {len(context)} items")
    for rank, line in context:
        print(f"  [rank {rank}] {line}")

# Output:
# budget 40: 28 tokens, 4 items
#   [rank 1] The site certificate renews every ninety days.
#   [rank 3] The site depends on the authentication service.
#   [rank 4] The backup starts every night.
#   [rank 2] Renewal runs the command provided by the certificate manager.
# budget 14: 14 tokens, 2 items
#   [rank 1] The site certificate renews every ninety days.
#   [rank 2] The site depends on the authentication service.

With budget 40 four entries get in, and the one at rank 2 ends up at the bottom: rank 1 opens the context and rank 2 closes it, while the least relevant sit in the middle. The duplicate about the certificate was discarded.

With budget 14 the second candidate by score (9 tokens) does not get in, because the space left is 7, and a less relevant one moves up: it is the greedy choice of the listing, and an alternative would be to stop at the first one that does not fit. Which is better depends on whether, for you, an incomplete context is worse or one that includes second-tier material. The weight of the graph (0.5) too is an example value, to be calibrated on questions of your own.

Five production aspects remain that tutorials leave for the end:

• privacy: what the agent remembers may contain personal data, and you need a verified deletion path;
• aging: an old fact does not disappear by itself, it must be closed (as in the graph) or devalued;
• conflicts: two facts that contradict each other require a precedence rule, for example the most recent wins, with a log of the changes;
• observability: how many retrievals fall below the threshold, how many tokens each part of the prompt takes up;
• fallback: if the memory does not answer, the agent must continue with the current conversation alone, saying so.

They are independent of one another, but the first three change the data model and are expensive to add later, while the last two can be added at any time: that is why it is better to decide right away how a memory is deleted and how it ages. Defense against abuse, for example hostile text that someone manages to get into the memory so that the agent rereads it later, falls under the risks covered in Security and Safety. How the agent uses the tools that read and write the memory is instead the subject of Advanced Tool Calling.

Frequently asked questions

What you can do now

You can tell session memory from memory across sessions, and the three types of content that the documentation calls semantic, episodic and procedural. You can keep a conversation window without losing the constraints, split a document into chunks with overlap knowing why the model's limit matters, read the score of a retrieval and reject one that is too low.

You can represent facts and relationships in a graph with a validity in time, answer multi-hop questions, and assemble the context inside a budget, putting the best results at the extremes. And you know that the figures of a project like Mem0 are to be read as measurements by its authors.

Deciding what to remember leaves two questions open: what it costs to carry the context around, which we compare in FinOps: Cost Optimization for Agents and LLM Tokens, and where the state lives when the process is stopped and restarted, as in production deployment.