AI Agent Memory Layer Compared: Mem0 vs Letta vs Zep vs LangGraph Memory

Every one of these four tools will tell you they've solved agent memory. None of them have solved retrieval ranking, and none of them have solved staleness. What they've actually done is move the RAG chunking argument from 2023 up one layer of abstraction, wrapped it in a cleaner SDK, and given it a name that sounds more finished than it is.

At a glance: MongoDB-backed Mem0 does LLM-driven fact extraction on top of vector recall. Letta (formerly MemGPT) gives the agent an OS-style memory hierarchy it manages itself. Zep builds a temporal knowledge graph so facts carry validity windows instead of flat similarity scores. LangGraph treats memory as a byproduct of checkpointed graph state rather than a dedicated product. The decision axes that actually matter: staleness tolerance, compliance exposure on persisted PII, engineering maturity for graph or OS-style infrastructure, and whether you're already locked into an ecosystem.

Platform	Pricing Model	Architecture	Best For
Mem0	Open source + managed tier	LLM fact extraction + vector store	Teams bolting memory onto an existing agent with minimal glue code
Letta	Open source + managed tier	Tiered core/archival memory, self-editing	Self-hosted stacks that want agent-controlled memory paging
Zep	Open source (Graphiti) + managed	Temporal knowledge graph	Enterprise use cases where facts change over time (plan tier, role, status)
LangGraph Memory	Free with LangGraph	Checkpointer-based state persistence	Teams already committed to the LangChain ecosystem

What Is an AI Agent Memory Layer, and Why Is It Replacing RAG?

An AI agent memory layer is infrastructure that persists facts, summaries, and interaction history across sessions, rather than re-embedding a static corpus on every call. It answers a different question than retrieval-augmented generation does: not "what document is relevant to this query" but "what did we already learn about this user, and is it still true."

From Single-Shot Retrieval to Persistent State

Classic RAG is stateless by design. You query, embed, retrieve top-k chunks, generate, and discard. Multi-step agents break that model because they run tool calls across turns and sessions, and they need durable state that survives the boundary between one conversation and the next. A support agent that forgets a customer downgraded their plan two weeks ago isn't doing RAG wrong, it's missing the state layer entirely.

Where the Term 'Memory' Gets Overloaded

"Memory" in agent marketing is a catch-all covering at least three distinct technical approaches: vector similarity recall, structured fact or entity extraction, and rolling LLM-generated summarization. Vendors rarely specify which one they mean, and some products blend all three without disclosing the mix. This matters for compliance too: once facts persist outside the original conversation, you have a durable PII surface with its own audit trail obligations, not just an engineering convenience.

Why Does Every Vendor Implement Agent Memory Differently?

Every vendor implements agent memory differently because the three underlying retrieval strategies trade off cost, latency, and accuracy in incompatible ways, and no single approach wins on all three axes. Vector recall is cheap and fast but noisy. Fact extraction is precise but expensive. Summarization is compact but lossy.

Vector Recall vs Structured Facts vs Summarization

• Vector recall: embed the interaction, store it, retrieve by cosine similarity at query time. Cheap, fast, but prone to retrieving semantically similar-but-outdated content with high confidence.
• Structured fact extraction: an LLM call parses each turn into discrete facts (entity, attribute, value) stored in a database or graph. More precise, but adds an LLM call per turn and depends entirely on extraction quality.
• Rolling summarization: periodically compress conversation history into a running summary injected into context. Compact, but lossy by construction, since anything not captured in the summary is gone.

This is close to a rerun of the 2023 RAG chunking debate: chunk size versus recall precision, overlap versus token cost. The argument didn't go away, it just moved one abstraction layer up into the memory SDK. Nobody selling a memory product has solved ranking or staleness at the retrieval layer. They've built a new interface around the same open problem.

How Does Mem0 Handle Agent Memory?

Mem0 handles agent memory through LLM-driven fact extraction backed by a vector store, designed to drop into an existing agent stack with minimal glue code. It runs an extraction step on each interaction, distills it into discrete memory objects, and stores those for retrieval on future turns rather than replaying full conversation history.

Architecture and Extraction Pipeline

Mem0's own documentation cites reductions in token usage and context size compared to full-history replay, a claim worth noting as vendor-published rather than independently benchmarked. The pipeline itself is straightforward: extract, embed, store, retrieve on relevance. That simplicity is the selling point.

Where Mem0 Fits

The honest limitation is that extraction quality depends entirely on the underlying LLM's summarization judgment, and errors compound silently across sessions since there's no ground-truth check on what got extracted. Teams running Mem0 in production commonly pair it with a document store like MongoDB for persistence and need observability tooling like Honeycomb to trace what got retrieved on a given turn and why, because without that trace, a wrong answer looks identical to a correct one from the outside.

• Pick Mem0 if you want the lowest integration overhead and can tolerate occasional extraction drift.
• Skip it if your use case requires provable fact provenance for audit purposes.

How Does Letta (Formerly MemGPT) Handle Agent Memory?

Letta handles agent memory through an operating-system-style hierarchy: a small in-context "core memory" block the agent always sees, and a larger "archival memory" store the agent pages content into and out of on its own initiative. The agent, not a fixed retrieval function, decides what stays resident.

Tiered Memory and the OS Analogy

The OS analogy is not just marketing framing, it's architecturally real. Letta's agent issues explicit memory-management function calls, similar to how a process requests paged memory from a kernel. That gives more transparency into why a fact was surfaced, since you can inspect the paging decision, but it also introduces nondeterminism: two runs given identical input can retrieve different context because the paging decision itself is a generative step, not a fixed lookup.

Where Letta Fits

Letta's self-editing memory blocks are a genuinely different design from a bolt-on vector cache sitting beside an otherwise unmodified agent loop. That distinction matters most for teams building on self-hosted models like Llama, where you control the full inference loop and can tune the memory-management prompts directly rather than treating them as a vendor black box. The honest criticism: agentic memory management adds real latency and token cost per turn compared to a straightforward retrieval call, since the agent has to reason about memory operations as part of every response cycle.

• Pick Letta if you're self-hosting and want the agent's memory decisions to be inspectable and tunable.
• Skip it if predictable per-turn latency matters more than architectural transparency.

How Does Zep Handle Agent Memory?

Zep handles agent memory through a temporal knowledge graph, built on its open-source Graphiti engine, that tracks when facts were true rather than treating every stored memory as equally current. A flat vector store has no native concept of "this was true then, but is it true now"; Zep's graph does.

Temporal Knowledge Graphs vs Flat Vector Stores

This distinction is not academic in enterprise deployments. A customer's plan tier changes, an employee's role changes, a support ticket's status changes. A vector store returns the semantically closest match regardless of when it was written, which means a stale fact with a high similarity score gets surfaced with the same confidence as a current one. That's how you get an agent that confidently tells a customer they're still on a trial plan three months after they upgraded.

The honest tradeoff is that graph construction and traversal add real engineering complexity and infrastructure surface area compared to a simple embedding lookup. You're not just storing vectors, you're maintaining entity resolution, edge validity windows, and graph query performance at scale. Of the three vendor approaches here, Zep comes closest to actually addressing staleness rather than just storing more context, but it demands more operational maturity, comparable to standing up a proper transformation layer like dbt for a data warehouse instead of running ad hoc SQL against raw tables.

• Pick Zep if stale facts produce real business or compliance risk in your domain.
• Skip it if your team doesn't have graph infrastructure experience and can't absorb the operational overhead.

How Does LangGraph's Built-In Memory Compare to Standalone Vendors?

LangGraph's built-in memory is a byproduct of its checkpointer-based graph state persistence, not a dedicated retrieval product. State gets saved at each graph step and reloaded on resumption, which gives you continuity within a graph's execution but not a purpose-built memory retrieval system.

For teams already committed to the LangChain ecosystem, this is the right default: it's free, it's already wired into the framework you're using, and it avoids adding a fourth vendor to your stack for a problem you can partially solve with what you have. The weakness shows up with cross-application memory sharing, since checkpointed state is scoped to the graph's execution context rather than to a user or entity across applications. If two separate agents built on two separate graphs both need to know that a customer downgraded, LangGraph's native memory won't bridge that gap without you swapping in a vendor backend behind the checkpointer interface.

The build-versus-buy calculus here is simple: LangGraph memory is free if you're already in that ecosystem, but you inherit its retrieval limitations, flat, execution-scoped, no staleness handling, unless you deliberately swap in Mem0, Zep, or a custom store as the checkpointer's backing implementation.

Which Agent Memory Layer Should You Choose for Your Compliance Requirements?

The memory layer you should choose depends first on your compliance obligations, specifically SOC 2 Type II attestation status, GDPR data residency, HIPAA suitability, and self-hosting availability, not on which SDK has the fastest quickstart. Persisted memory is a durable PII surface, and that changes the vendor evaluation criteria.

Framework Coverage Matrix

Platform	SOC 2 Type II	GDPR Residency Controls	HIPAA Suitability	Self-Hosting
Mem0	Managed tier only, verify current status directly	Depends on chosen vector store backend	Requires BAA review with vendor	Yes, open source core
Letta	Verify with vendor for managed tier	Full control if self-hosted	Feasible self-hosted with model control	Yes, designed for it
Zep	Verify with vendor for managed tier	Depends on graph store deployment region	Requires BAA review with vendor	Yes, Graphiti is open source
LangGraph Memory	Inherits your own infrastructure's attestation	Full control, scoped to your checkpointer backend	Feasible if backend is your own compliant store	Yes, fully self-managed

Data Residency and Right-to-Erasure Gaps

A user's deletion request under GDPR Article 17 has to propagate to the memory layer, not just to the visible chat log. If a fact was extracted, embedded, and stored in a separate vector index or graph node, deleting the conversation transcript does nothing to that derived record unless your pipeline explicitly cascades the deletion.

Most teams have not mapped which memory store fields count as personal data under their DPA, and vendor documentation on this point is thin across all four platforms as of this writing. Treat every extracted fact, embedding, and graph node as personal data by default until your legal team says otherwise.

Pair whichever memory layer you choose with disciplined secrets management, such as 1Password, for the API keys these systems require, and enforce consistent data residency configuration across environments using infrastructure-as-code like HashiCorp Terraform. Manual, console-driven deployment of a memory store's regional configuration is how you end up with EU customer data quietly replicated to a US region during an unremarkable staging deploy.

How Do You Test Whether an Agent Memory Layer Actually Works?

You test whether an agent memory layer actually works by building a regression suite that injects contradictory fact updates over time and measures whether the agent retrieves the current version or a stale one, not by trusting a vendor's demo notebook.

Building a Memory Regression Suite

1. Define canonical fact-recall test cases: a set of facts the agent should know (plan tier, account status, prior support tickets).
2. Inject an update that contradicts the original fact, at a known point in the session or conversation history.
3. Query the agent after the update and check whether it returns the current fact, the stale fact, or a hedged non-answer.
4. Score the result against ground truth, not against a vague "sounds reasonable" heuristic.
5. Run the suite on every memory-layer change, prompt change, and model version bump, not just once at initial integration.

An eval framework like Promptfoo can run these fact-contradiction scenarios as automated regression tests, treating stale-fact retrieval the same way you'd treat a broken unit test rather than a one-off manual QA pass.

Context rot in memory layers is silent. The agent doesn't error out, it doesn't return an empty result, it answers confidently with an outdated fact. That makes it worse than a RAG miss, which at least fails visibly by returning nothing relevant.

Log retrieval traces, what was fetched, from which store, with what confidence score, to an observability platform such as Sentry or a Grafana dashboard, so retrieval failures are visible to your team before a customer reports being told something that hasn't been true for weeks.

What Should You Do Before Adding a Memory Layer to Your Agent Stack?

Audit what facts actually need to persist across sessions versus what can simply be re-derived per session from source systems, before you evaluate a single vendor. Not every agent needs full episodic memory, and adding one you don't need only adds compliance surface without corresponding benefit.

Vendor choice should be driven by your staleness tolerance and your compliance obligations, not by which SDK produced the smoothest quickstart demo. A support agent handling billing status needs Zep-grade temporal accuracy. An internal documentation assistant answering static-content questions probably doesn't need a memory layer at all, just better retrieval on a fixed corpus.

Before routing any production traffic through Mem0, Letta, Zep, or a vendor-backed LangGraph checkpointer, run a two-week shadow test: log every memory retrieval alongside ground truth for the same query, measure staleness and extraction error rates, and only promote the memory layer to production once you can quantify how often it's confidently wrong.