Do AI Agents Need a Memory Layer? Mem0 vs Letta vs Zep

Updated 28 September 2026. The first version of this post compared these tools with latency figures I had not measured, and one of them was badly wrong. I have since run Mem0, Letta, Zep, LangMem and GoodMem through the same benchmark, so the comparison below is now measured. The full method, harness and per-question log are in the benchmark write-up ↗.

The category got real money behind it in late 2025. Mem0's $24M Series A and AWS picking it as the exclusive memory provider for its Agent SDK turned agent memory from a research idea into a funded product category. By April 2026 the field had consolidated to four names: Mem0, Letta, Zep with its Graphiti engine, and LangMem. Mid-2026 is the moment teams feel forced to pick. Most of them should not pick anything yet.

What is an agent memory layer?

An agent memory layer is a service that extracts, consolidates, decays, and retrieves long-term facts about users and agents across sessions, beyond what fits in the context window. It is not RAG over documents. RAG retrieves knowledge from a corpus you control; memory accumulates facts about a relationship over time.

That one-sentence definition hides four distinct jobs:

• Extraction. During or after a conversation, an LLM pass pulls out durable facts: "user prefers concise answers", "user's company has 12 locations".
• Consolidation. New facts get merged with old ones. "User is vegetarian" plus a later "user ordered the chicken" needs conflict resolution, not two contradictory rows.
• Decay. Facts age out or get invalidated. "User lives in Austin" stops being true the day they move.
• Retrieval. On every turn, the layer injects the handful of facts relevant to the current message.

The contrast with RAG matters because teams keep buying memory products to solve retrieval problems. If your agent needs to answer questions from your docs, that is retrieval over a corpus, the kind of RAG system I build for clients ↗ at $4k to $12k for an MVP. I covered where retrieval itself is heading in my piece on agentic retrieval ↗. Memory is the other axis: not what the agent knows, but what it remembers about you.

Do you actually need one?

You need a memory layer only if you answer yes to at least one of three questions: multi-session personalization, shared memory across parallel agents, or consolidation and conflict resolution. If the answer to all three is no, close the vendor tabs. A Postgres table covers you, and I will show the exact schema below.

Here is the framework as I actually run it:

1. Do users return across sessions and expect the agent to remember them? A support bot that should recall last week's ticket: yes. A one-shot document summarizer: no.
2. Do multiple agents or parallel sessions need to share one memory state? If agent A learns something agent B must act on within minutes, you have a shared-state problem that a naive table handles badly.
3. Do memories need consolidation or decay? If facts about a user change and contradict over time, and stale facts cause wrong behavior, you need conflict resolution logic you probably should not write yourself.

If you can't name a consolidation or decay requirement, you don't need a memory vendor; you need a Postgres table with an embedding column.

That sentence has saved several of my consulting conversations from a premature vendor evaluation.

Why is Mem0 the default pick in 2026?

Mem0 is the funding-validated default. It raised $24M total (Series A led by Basis Set, announced October 28, 2025, with Y Combinator and Peak XV participating), has 48,000+ GitHub stars, and is the exclusive memory provider for AWS's Agent SDK; its API calls grew from 35M to 186M during 2025. As of June 2026, if you want a hosted memory API with an ecosystem that meets you halfway, Mem0 is the safe call.

Architecturally, Mem0 sits beside your agent as an API. You call add() with conversation messages and it runs LLM extraction, deduplication, and storage into a vector store with an optional graph layer; you call search() per turn to fetch relevant memories. The Node SDK (mem0ai on npm) is mature by JavaScript standards, with TypeScript types and near parity with the Python client for core operations, which was not true of every competitor when I evaluated in early 2026. Native integrations in CrewAI, Flowise, and Langflow mean low-code teams get it almost for free.

The trade-off to understand: the hosted platform is the smooth path, while self-hosting the Apache 2.0 core means operating your own vector store and paying for the extraction LLM calls yourself. Either way, every memory write is an LLM call somewhere. Budget for it. In my benchmark that extraction came to 1.5 seconds per turn written, against under 0.4 seconds for the two systems that call no model while writing.

What makes Letta different?

Letta is the MemGPT lineage: memory the agent itself owns and edits. Instead of a sidecar API skimming facts in the background, Letta gives the agent named memory blocks that it rewrites with its own tool calls. In April 2026 Letta shipped Letta Code and a Conversations API for shared memory across parallel sessions.

Letta is heavier conceptually because it is closer to an agent runtime than a library. You adopt its server, its agent abstractions, and its view that memory editing is a tool the agent calls when it decides a fact matters. That is genuinely useful when the agent should reason about what to remember rather than have facts extracted behind its back. The Conversations API answers my second framework question directly: parallel sessions of one agent reading and writing shared memory blocks without trampling each other. If your architecture is a fleet of workers acting as one agent, Letta has the strongest answer of the three. The cost is tokens and complexity: the agent spends part of its budget managing memory, and you debug memory behavior as agent behavior.

Measured on its archival memory alone (version 0.11.7), Letta came third of six configurations at 52.2% recall, and it calls no model at all while writing. It also tied for the best multi-hop score, the questions that need two facts from different sessions.

When does Zep's temporal knowledge graph win?

Zep wins when your facts are entities with relationships that change over time. Its Graphiti engine builds a temporal knowledge graph: facts carry valid-from and valid-to timestamps, so "user moved from Austin to Denver" invalidates the old city without deleting history. For CRM-style agent state, it is the strongest model here.

Where Mem0 stores discrete memory entries, Graphiti stores entities and edges with explicit time bounds. When a new fact contradicts an old one, the old edge gets an invalid-at timestamp instead of being overwritten, so the agent can answer both "where does this customer live" and "where did they live when they signed the contract". If your agent serves sales, support, or account management workflows where the state of record is people, companies, and subscriptions that evolve, Zep models that more honestly than flat entries. The graph build adds write-path cost, and for simple preference recall it is more machinery than you need.

The benchmark was hard on Zep: last on recall at 37.0%, 3.65 seconds of graph building per turn, and only 8 of 22 temporal questions right, which is the category its graph is built for. What it does well is brevity. It returned 212 tokens of context per answer, the least of any system, and refused unanswerable questions as reliably as anyone. Choose it for what a small context saves you on every query, not for coverage.

How do Mem0, Letta, Zep, LangMem and GoodMem compare?

Measured on the same conversation, the best of five memory systems answered 57.6% of questions correctly, and none reached the 80 to 93% that vendors publish. In September 2026 I ran all five on one LoCoMo conversation of 419 turns, with 120 questions, one shared answering model (gpt-4o-mini at temperature 0) and the top 10 results from every system. Each one is graded only on the text it retrieved, so the answering model cannot fill gaps from its own knowledge.

Recall is graded by an LLM judge. A stricter reading lowers every row by about 2 to 4 points and changes no positions. "Refused unanswerable" is the share of 28 trick questions with no answer in the conversation that the system correctly declined, so higher is better. Context tokens is the size of what each system hands the model per answer, and write per turn is the time to store one turn of conversation. GoodMem was tested on GoodMem Cloud, so its search time includes a network round trip; the others ran on my machine. GoodMem also installs on your own server with one command, and that setup was not measured.

What the table changed in this post:

• My old latency figure for Mem0 was wrong. The first version said Mem0 added 150 to 300 ms per turn. Its measured median search is 38 ms, the fastest of the five, partly because it ran on local Qdrant.
• No single column decides it. I used to call latency the deciding row. On real numbers the fastest system is mid-table on recall, the most accurate is the slowest, and the cheapest to read is the least accurate.
• Extracting facts on every turn did not pay for itself here. GoodMem and Letta call no model while writing and finished first and third. Mem0, LangMem and Zep each run an extraction model on every turn, spend 1.5 to 4.2 seconds doing it, and scored lower.
• The recall leader answers more trick questions. GoodMem's vendor config, which adds a reranker, tops recall but refuses fewer unanswerable questions than its own defaults. Decide which failure costs you more.

Plain Postgres is not in the table because I have not run it through the harness. Its search is one indexed query in a database you already have, so it will be quick, but its recall on this workload is unmeasured, and I am not going to put a guessed number in a table again.

Here is how the six options differ in shape, which is the part that should narrow your list before any number does:

On licences: Mem0, Letta and Zep's Graphiti engine are Apache 2.0, LangMem is MIT, pgvector uses the PostgreSQL licence, and GoodMem is not open source but free to self-host commercially.

Measure in your own stack before deciding. Your conversations are not LoCoMo, and a memory call sits on the critical path of every turn. I trace per-turn latency with the OpenTelemetry setup from my agent observability guide ↗.

When is plain Postgres with pgvector enough?

Plain Postgres with pgvector is enough when you are single-tenant or under roughly 10k users, you need simple preference recall rather than consolidation, and your facts rarely contradict each other. The cost is $0 in extra infrastructure if you already run Postgres, and retrieval is one indexed query in a database you already operate.

The entire schema:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE memories (
  id          BIGSERIAL PRIMARY KEY,
  user_id     TEXT NOT NULL,
  content     TEXT NOT NULL,
  embedding   VECTOR(1536) NOT NULL,
  category    TEXT NOT NULL DEFAULT 'general',
  updated_at  TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX memories_user_idx ON memories (user_id, category);
CREATE INDEX memories_embedding_idx ON memories
  USING hnsw (embedding vector_cosine_ops);

And the retrieval function, the whole "memory layer" in about 30 lines of Node:

import pg from "pg";
import OpenAI from "openai";

const pool = new pg.Pool({ connectionString: process.env.DATABASE_URL });
const openai = new OpenAI();

export async function recallMemories(userId, query, opts = {}) {
  const { category = null, limit = 5, minSimilarity = 0.55 } = opts;

  const { data } = await openai.embeddings.create({
    model: "text-embedding-3-small",
    input: query,
  });
  const embedding = `[${data[0].embedding.join(",")}]`;

  const { rows } = await pool.query(
    `SELECT content, category, updated_at,
            1 - (embedding <=> $2) AS similarity
       FROM memories
      WHERE user_id = $1
        AND ($3::text IS NULL OR category = $3)
      ORDER BY embedding <=> $2
      LIMIT $4`,
    [userId, embedding, category, limit]
  );

  return rows.filter((r) => r.similarity >= minSimilarity);
}

The write path is one LLM call at session end that extracts facts as JSON and upserts them. Getting that JSON reliable is its own topic; I compared the options in my structured outputs piece ↗.

My own production agent needs even less than this. The review-reply product I run keeps each business's reply tone and preferences as ordinary columns on its settings row and reads them on every reply. No vector search and no memory vendor, because nothing about a business's tone needs to be found by similarity; it only needs to be looked up. When I scope agent and automation projects for clients ↗, a table like the one above is my default, and a memory vendor enters the picture the day memories need consolidation and decay across multiple agents, not before.

Which memory option fits your app shape?

Map your app shape to the tree below. The branches encode the three framework questions, the entity-versus-preference distinction, and what the benchmark showed each system is actually good at. Most of the agent builds I scope land on the Postgres leaf, and that has held even after the 2025 funding wave made the vendors easy to adopt.

Do returning users expect the agent to remember them?
|
+-- No  -> skip the memory layer (context window + session store)
|
+-- Yes
    |
    +-- Recall matters most? Runs on GoodMem
    |   Cloud or on your own server.           -> GoodMem
    |
    +-- Agent should own and edit its memory,
    |   or parallel sessions share state?      -> Letta
    |
    +-- Search latency matters most, or you want
    |   broad framework integrations?          -> Mem0
    |
    +-- Entities that change over time, and
    |   context size matters more than
    |   coverage?                              -> Zep (Graphiti)
    |
    +-- Simple preference recall, <10k users,
        no consolidation or decay?             -> Postgres + pgvector

Start at the bottom leaf and earn your way up. Migrating a memories table into Mem0 or Zep later is a weekend of work; unwinding a vendor dependency you never needed is much worse.

Key takeaways

• An agent memory layer extracts, consolidates, decays, and retrieves long-term facts across sessions. It is not RAG over documents, and buying one to fix retrieval is the most common mistake I see.
• Run the three-question test: multi-session personalization, cross-agent shared state, consolidation and conflict resolution. Zero yes answers means you do not need a vendor.
• Measured on one 419-turn conversation and graded only on what each system retrieved, no memory system cleared 58%: GoodMem (vendor config) 57.6%, GoodMem (defaults) 53.3%, Letta 52.2%, Mem0 50.0%, LangMem 45.7%, Zep 37.0%. Plan for about half your questions answered correctly, not the 80 to 93% on vendor pages.
• Mem0 is the funded default: $24M raised, 48,000+ GitHub stars, exclusive memory provider for AWS's Agent SDK. Measured, it has the fastest search of the five at 38 ms and mid-table recall.
• Letta fits agents that own and edit their memory, and it came third of six configurations while calling no model to write. Zep returns the smallest context at 212 tokens but had the lowest recall.
• The two systems that skip per-turn fact extraction, GoodMem and Letta, both beat the three that pay for it.
• A Postgres table with a pgvector column costs $0 extra on a database you already run. I have not benchmarked it yet, so its recall is unmeasured. Its cost and simplicity are not.