Skip to main content

01 - Letta (ex-MemGPT): Persistent Memory for AI Agents

How an agent remembers beyond the context window: Letta memory blocks, archive, MemFS and the Agent SDK, with a model you can run.

What do you think about this article?

01 - Letta (ex-MemGPT): Persistent Memory for AI Agents
Agentic PlatformsPhoto:Pavel DanilyukPexels

At a Glance

How an agent remembers beyond the context window: Letta memory blocks, archive, MemFS and the Agent SDK, with a model you can run.

The problem: an agent that starts from zero in every conversation

A language model remembers nothing between one call and the next: it sees only the text you send it, which is the context window. If Alice tells you about her project on Monday and comes back on Thursday, the agent knows who she is only if you send back everything that was said on Monday. Sending everything back costs tokens on every turn, and sooner or later it no longer fits. Letta, the current name of MemGPT, was born for this: the agent itself decides what to keep in the prompt, what to archive and when to go and fetch it again.

The short answer is that Letta's persistent memory is made of two things. There are a few small blocks of text that are always present in the prompt and say who the agent is and who the user is. And there is an archive outside the prompt, which the agent queries with a search when it needs something. The part that changes fastest is everything else: the product, in November 2026, is no longer the one described by many tutorials, and we will see that with the sources in hand.

The MemGPT idea, in a model you can run

The idea comes from the paper MemGPT: Towards LLMs as Operating Systems (Packer et al., arXiv 2310.08560, October 2023). The authors propose virtual context management: just as an operating system makes memory look large by moving data between fast and slow storage, an agent can appear to have unlimited context by moving information between the prompt and an archive. The Letta blog post on memory blocks (14 May 2025) describes the concrete form: two blocks in the prompt, human and persona, which the agent can rewrite and which have a character limit.

The three memory zones, in the order in which the text moves away from the model, are these:

  • Blocks in the prompt: small, with a character cap, rewritten by the agent; the model sees them on every turn.
  • Message window: the recent conversation; when it exceeds the budget, the oldest messages leave the prompt.
  • Archive: everything that has left, plus whatever the agent chose to save; it can be retrieved only with an explicit search.

The criterion that holds the list together is the cost of presence: every character in the prompt is paid for on every turn, every character in the archive is paid for only when you search for it. That is why the blocks have a cap: without one, the agent would inflate them until it rebuilt the very problem it was supposed to solve. The order of the list is not random: it runs from the zone closest to the model to the farthest, and the cost drops at every step.

To see the mechanism without a key, without a model and without a server, I wrote a teaching model in Python. It is not Letta code and it does not imitate it line by line: the search is an overlap of words, not an embedding, and the window is measured in words, not tokens. It makes three behaviors visible: the rejection of a block that is too long, the eviction of old messages into the archive, and the retrieval with a search.

Memory hierarchy in miniature (teaching model, not Letta)
"""Teaching model of the MemGPT memory hierarchy: NOT Letta code."""
import re


class BlockLimit(Exception):
    pass


class Memory:
    def __init__(self, block_limit=120, context_budget=60):
        self.blocks = {"human": "", "persona": "Technical assistant, concise answers."}
        self.block_limit = block_limit          # characters per block, as in the original MemGPT
        self.context_budget = context_budget      # words allowed in the message window
        self.window = []                          # recent messages, in the context
        self.archive = []                          # external memory, outside the context
        self.summary = ""

    # --- "core" memory: always in the prompt, small, editable by the agent
    def block_append(self, name, text):
        new_text = (self.blocks[name] + " " + text).strip()
        if len(new_text) > self.block_limit:
            raise BlockLimit(f"{name}: {len(new_text)} characters, limit {self.block_limit}")
        self.blocks[name] = new_text

    def block_replace(self, name, text):
        if len(text) > self.block_limit:
            raise BlockLimit(f"{name}: {len(text)} characters, limit {self.block_limit}")
        self.blocks[name] = text

    # --- "archival" memory: outside the context, queried with a search
    def archive_insert(self, text):
        self.archive.append(text)

    def archive_search(self, query, k=2):
        words = set(re.findall(r"\w+", query.lower()))
        scores = [(len(words & set(re.findall(r"\w+", t.lower()))), -i, t)
                    for i, t in enumerate(self.archive)]
        return [t for p, _, t in sorted(scores, reverse=True)[:k] if p > 0]

    # --- message window: when it overflows, the oldest go to the archive
    def message(self, text):
        self.window.append(text)
        while sum(len(m.split()) for m in self.window) > self.context_budget:
            evicted = self.window.pop(0)
            self.archive_insert(evicted)
            self.summary = f"[{len(self.archive)} messages in archive]"

    def prompt(self):
        blocks = "\n".join(f"<{n}>{t}</{n}>" for n, t in self.blocks.items())
        return f"{blocks}\n{self.summary}\n" + "\n".join(self.window)


m = Memory()
m.block_append("human", "Alice, software engineer.")
try:
    m.block_append("human", "Prefers detailed technical answers, with code examples, " * 2)
except BlockLimit as e:
    print("rejected:", e)
m.block_replace("human", "Alice, software engineer. Wants code examples.")

for i in range(1, 7):
    m.message(f"turn {i}: " + "word " * 9)
m.message("turn 7: the React bug was a useless re-render in the Table component")
print("window:", len(m.window), "messages | archive:", len(m.archive), "messages")
print("search while the message is still in the window:", m.archive_search("React bug re-render"))
m.message("turn 8: " + "word " * 50)
print("window:", len(m.window), "messages | archive:", len(m.archive), "messages")
print("search after eviction:", m.archive_search("React bug re-render", k=1))
print("line in the prompt:", m.summary)

# Output:
# rejected: human: 137 characters, limit 120
# window: 5 messages | archive: 2 messages
# search while the message is still in the window: []
# window: 1 messages | archive: 7 messages
# search after eviction: ['turn 7: the React bug was a useless re-render in the Table component']
# line in the prompt: [7 messages in archive]

Three lines matter. block_append rejects the text that would push the block beyond block_limit and raises BlockLimit: in the real system it is the error that forces the agent to rewrite the block instead of appending to it. message is the heart of the eviction: as long as the words in the window exceed the budget, the oldest message moves into archive. archive_search sorts by shared words, with the message index as a tie-breaker, so the output is the same on every run.

The output tells the sequence: the second insertion into human is rejected because it brings the block to 137 characters against a limit of 120. After seven messages the window holds five and the archive two, and the search for the React bug finds nothing, because that message is still in the prompt and there is no need to search for it. Only after the long message of turn 8 does turn 7 leave the window, land in the archive, and the same search returns it. When not to use this scheme: if the conversation always fits in the context, the archive is useless complexity.

What Letta is today

Here the text changes register, because the facts have changed. The README of letta-ai/letta now says that the active code lives in letta-ai/letta-code, and that the earlier API server, the V1 one, has been retired and sits in the archive branch. The letta-ai package cited by many tutorials does not exist on PyPI (the request to the registry answers 404); letta exists, and its latest version, 0.34.12 of 12 October 2026, presents itself as "Letta Code: stateful agents in your terminal". Letta Code and the Agent SDK are Apache-2.0, like the historical repository.

AspectIn the V1 tutorialsToday (letta-code and Agent SDK)
Installationpip install and Docker on the repositorynpm install -g @letta-ai/letta-code, or uv tool install letta
ServerREST API server to startApp Server, letta server --backend local
ProgrammingREST client (Python too, letta-client)Agent SDK in TypeScript, @letta-ai/letta-agent-sdk
Where memory livesblocks and archive in the server's databaseMarkdown files in the agent's git repository (MemFS)

Read the table as a warning, not as a judgment: if a tutorial has you write LocalClient, run letta db init or build a Docker image from the repository, it is describing the retired server. The source to follow is the current documentation, docs.letta.com. There is also a default to know about: the Letta Code README says that Letta Cloud is the default, and that on first launch you choose whether to sign in or proceed locally. Whoever keeps the data in-house must choose local explicitly, with letta backend local or --backend local.

Where memory lives now: MemFS and dreaming

In Letta Code, memory is a set of Markdown files with a YAML header, inside a git repository that belongs to the agent. The file name is the block label: the persona memory becomes persona.md. The documentation calls this layer MemFS (also "context repository") and explains the rule that replaces the V1 character cap: the files at the root enter the prompt on every turn, while directories with their own MEMORY.md stay out until the agent decides to read them.

Structure of a memory, from docs.letta.com/concepts/memfs
$MEMORY_DIR/
├── MEMORY.md
├── persona.md
├── human.md
├── reference/
│   ├── MEMORY.md
│   └── project-notes.md
└── skills/
    └── my-skill/
        └── SKILL.md

The mapping to the teaching model is direct: persona.md and human.md correspond to the blocks in the prompt, the reference/ directory to the archive, and its MEMORY.md to an index that says where to look. Two practical consequences. First, MemFS does not include a semantic or vector index: the documentation says that agents search with the ordinary search and read tools on files, and that semantic search requires a separate mod that in turn depends on QMD. Second, every change is a commit: you have the history, and you can go back.

Dreaming is the part that is new compared with the paper. These are background sub-agents that reread recent conversations, consolidate what is worth keeping and update memory without interrupting the work. They start after a number of steps or when the context is compacted. One option has the agent itself review the proposals in a second conversation: the documentation warns that it uses more of the model's tokens and does not ask for your approval.

An agent with the Agent SDK (a sketch to adapt, compiled but not run)

The listing below is the documentation's example adapted to Alice's case. I compiled it with tsc --strict against @letta-ai/letta-agent-sdk 0.8.86, the current version on the npm registry on 13 November 2026. I did not run it, because it requires an App Server listening and a model provider connected: it is a sketch to adapt. The compiler is still a real check: writing trigger: "step-conteggio" instead of "step-count" it rejects the type, and the fix shows up before execution. (The strings passed to persona and human and the question to the agent are Italian, as in the compiled listing: they are data for the model, not code.)

Letta Agent SDK: agent with persona, user and dreaming
import { LettaAgentClient } from "@letta-ai/letta-agent-sdk";

const client = new LettaAgentClient({
  backend: "remote",
  url: process.env.LETTA_APP_SERVER_URL ?? "http://127.0.0.1:4500",
  authToken: process.env.LETTA_APP_SERVER_TOKEN,
});

const agentId = await client.createAgent({
  persona: "Sei un assistente di ricerca. Annota ciò che impari sull'utente.",
  human: "Utente: Alice, ingegnera software, preferisce risposte concise.",
  dreaming: { trigger: "step-count", behavior: "auto-launch", stepCount: 25 },
});

await using session = client.createSession(agentId, { cwd: process.cwd() });
await session.send("Quale era il mio ruolo?");
for await (const message of session.stream()) {
  if (message.type === "assistant") console.log(message.content);
}

persona and human are shortcuts that set the two base memories; whoever wants full control passes instead a memory array of label and value objects, and each entry becomes a file. For dreaming the documentation lists trigger ("off", "step-count", "compaction-event") and behavior ("reminder" or "auto-launch"), which can be set only at creation. The page does not state the default value of trigger: to find out, do not guess, read the session's init message, which reports the effective settings.

The "remote" backend connects to an App Server that you run yourself, with a url and an authToken read from the environment; the App Server startup guide specifies that for any address other than loopback the WebSocket authentication must be configured (for example with --ws-auth capability-token and a file with the token). With "local" the SDK starts the App Server as a subprocess on the same machine. await using closes the session by itself; that is why the test tsconfig includes the ESNext.Disposable library. The for await loop receives the messages as a stream and here prints only the assistant's.

Self-hosting: what stays yours and what does not

The "Self-hosting" page of the documentation distinguishes two forms: the local runtime, in which the CLI runs the agents in its own process without needing a Letta account, and the App Server, which hosts the agents on an always-on machine and exposes them to the SDK. The commands are few and I report them as the documentation writes them, without having run them: starting them means opening a port and connecting a provider.

Local runtime with Ollama and App Server (commands from the documentation, not run)
npm install -g @letta-ai/letta-code
letta --backend local connect ollama
letta server --backend local --listen ws://127.0.0.1:4500

The first line installs the CLI (it needs Node.js 22.19 or later). The second connects an Ollama listening on the same machine as the model provider, and it is the choice that keeps inference local as well. The third starts the App Server bound to 127.0.0.1: it listens only on the machine itself, and for any address other than loopback the documentation asks you to configure authentication (and putting TLS in front is good practice, not a requirement that I verified). The documentation points out that the choice of model is separate: if you connect a remote provider, the prompts go to that provider.

The warnings on the same page, translated into obligations for whoever runs the service:

  • the built-in tools run where the agent runs, so on the App Server machine: the file and command permissions are those of that machine too;
  • self-hosted agents cannot be reached from chat.letta.com;
  • they have no automatic backup: copying the memory's git repository is your job.

The first point is the most delicate: an agent that runs commands on your machine has the permissions of its process, and must be treated like any service with shell access, with a dedicated user and no privileges. The third is the one people forget: locally the agent's memory is a repository on disk, so the backup is that of a folder, and it must be scheduled before the first day of use.

When persistent memory does damage

A memory that lasts is an archive of errors that last. If the agent writes a wrong fact, or a text taken from a hostile document, that fact re-enters the prompt of every future conversation: it is the same problem as prompt injection, but with a delayed effect. This reasoning is mine, not a measurement: the defense is to treat the memory files as untrusted input, reread them on a sample basis and limit the tools the agent can use after reading them.

Versioning has a less obvious side. Because every change is a commit, deleting a personal datum from the file does not delete it from the history: if you must honor a deletion request, you have to rewrite the repository's history, or not put in memory at all what you could no longer remove. Costs also grow silently, because the files at the root enter every prompt; the documentation offers the /doctor command (from the CLI) precisely to check duplicates and token usage in the system prompt.

Finally there is the risk of dependency, and Letta shows it well: the V1 API on which the tutorials rested has been retired. If you build a product, keep the SDK behind a thin interface of your own, pin the version (the SDK is at 0.8.86, below 1.0: expect changes) and read the changelog before upgrading. An agent with memory is the right choice when the user returns and must be recognized; it is oversized for a single request with no history.

What you can do now

You can explain why an agent needs a memory hierarchy and what each level costs, and you have a runnable model to reason with. You can recognize a tutorial built on the retired V1 server, choose between cloud and local before the first launch, and write an agent with the Agent SDK that the compiler has already checked. In the next article the same question, what the agent remembers and where its state lives, gets the answer of a TypeScript framework; the comparison of the three orchestrators and the cost analysis come after, and the article on debugging and traces closes the series with a way to see what an agent really does.

Frequently asked questions

Is Letta still open source?

Yes: the letta-ai/letta and letta-ai/letta-code repositories are published under the Apache-2.0 license according to the GitHub API, and the Agent SDK on npm also declares Apache-2.0. Letta Cloud, on the other hand, is a hosted service and not software you download, and it is the CLI's default.

Can I use Letta without an account and without the cloud?

Yes, with the local backend: the self-hosting documentation says that with the local runtime the agent's state (messages, memory, provider connections) stays on the device and no Letta account is needed. If you connect a remote model provider, however, the prompts reach that provider.

Are Letta and MemGPT the same thing?

MemGPT is the name of the research project and of the 2023 paper; Letta is the current name of the project and of the company, and the README calls it "f.k.a. MemGPT". The PyPI package pymemgpt stops at 0.3.25 of September 2024, while letta is updated to October 2026.

Sources

The version figures (letta 0.34.12, @letta-ai/letta-code 0.34.56, @letta-ai/letta-agent-sdk 0.8.86) come from the PyPI and npm registries, queried on 2026-11-13 with npm view and with the PyPI JSON API. The Python model was run with python3 -I twice, with the same output; the TypeScript listing was compiled with tsc in strict mode and is a sketch to adapt: no server was started and no model was called.

Try These Tools

Free developer tools related to this topic.

Commenti

Caricamento commenti...

Sign in to comment

New articles and tools on Telegram too

The Telegram channel flags the site's new articles and tools.

Did you like this article?

Explore other content on the blog or discover my projects