Agent Memory Is a Cross-Tenant Database. Partition It Like One

Admit Memory Is a Database You Didn't Design

Count the places an agent's memory actually lives. There's the obvious vector collection of past conversation chunks. Then the rolling session summaries, the 'user preferences' table someone added in week three, the embedding cache keyed on input text, the trace store your observability tool keeps for 30 days, and the eval dataset an engineer built by exporting interesting production sessions. Every one of those holds customer content. Usually only the first one got a design review.

The failure doesn't need anything exotic. OpenAI's March 2023 ChatGPT incident is the reference case. A bug in the redis-py client library let some users see other users' chat titles, and for a short window it exposed partial payment details for about 1.2% of ChatGPT Plus subscribers active in that period, according to OpenAI's own post-mortem. The fault sat in a caching layer. Agent memory adds more of those, and they get written faster and reviewed less.

My position is simple. If memory outlives a single request, it gets treated exactly like your primary database: a named owner, tenant keys, a retention policy, a way to delete it, and someone actually reviewing who has access. Teams push back because it slows the demo. It does. By about a week, and I'd take that week over the incident every time.

• →Inventory every store holding agent memory or anything derived from it, including caches, traces and eval exports.
• →Assign each store an owner and a data classification, the same as a production table.
• →Mark which stores hold raw customer text versus derived artifacts; both count.

Partition on Write and Let Storage Enforce It

The weakest isolation pattern I see proposed is a shared collection with a tenant_id metadata field and a retrieval call that filters on it. It works until one code path forgets the filter: a new tool, a background summariser, a debugging script someone promoted to a cron job. A metadata filter is a convention. Conventions get skipped.

Push isolation down to where it can't be skipped. In rough order of strength: a separate database or index per tenant, a separate namespace or collection per tenant with credentials scoped to it, or row-level security in Postgres with pgvector, where the database itself refuses rows outside the session's tenant. Per-tenant indexes cost more at the long tail of tiny customers, and you'll probably end up with a hybrid: dedicated indexes for enterprise accounts, namespaces with RLS for the rest. That's fine. A shared index with an application-level filter isn't.

The rule I'd hold hardest: the model never picks the tenant. Tenant scope comes from the authenticated session, gets bound to the memory client when the request starts, and the agent's tools receive an already-scoped handle. If a tool signature takes tenant_id as an argument the model fills in, a prompt injection can ask for a different one. The same reasoning applies to per-user permissions inside a tenant, which I've covered in why RAG leaks without document-level permissions.

Filter before retrieval, not after. Post-filtering a top-k result means the ranking already considered other tenants' vectors, and if the filter drops everything, some implementations quietly fall back to unfiltered results. Pre-filtering, or physically separate indexes, removes the question.

If a tool signature takes tenant_id as an argument the model fills in, a prompt injection can ask for a different one.

• →Bind tenant scope to the memory client at request start, from the auth token, never from model output.
• →Use storage-level isolation (separate index, scoped namespace credentials, or row-level security), not a metadata convention.
• →Pre-filter retrieval; treat any empty-result fallback to global search as a bug.
• →Write keys, cache keys and file paths with the tenant ID as the first component.

Turn Off Cross-Tenant Learning Until You Can Defend It

Raw conversation chunks are easy to partition. The leak path that worries me more is consolidation: a background job that reads many sessions and writes distilled 'lessons' or 'facts' the agent should remember. Some memory frameworks do this by default, extracting facts from conversations and merging them into a store. Run that job across tenants to make the agent 'smarter for everyone' and you've built a pipeline that launders one customer's specifics into shared context.

Here's a hypothetical. A support agent for a B2B tool learns from Customer A's sessions that 'the Q3 migration to the new billing provider broke SSO for the finance team'. The consolidator generalises it to 'SSO issues often follow billing migrations' and stores it globally. That sounds harmless until Customer B's agent volunteers it with enough detail that B can guess who A's billing provider is. No row was misfiled. The summariser did exactly what it was asked.

Every consolidation, summarisation and preference-learning job should default to a single tenant's partition. It reads there. It writes there. If the product genuinely needs cross-customer improvement, I'd treat that as a separate data product with its own pipeline: explicit contractual permission, PII and entity scrubbing, a human reviewing whatever gets promoted, and an output far smaller than people expect. Most teams asking for 'shared learning' really want better prompts and better retrieval over their own docs. Neither carries any tenant data.

• →Every background memory job declares the tenant it runs for; jobs without one fail to start.
• →Global memory contains only content your own team authored, such as product docs and policies.
• →Any cross-tenant promotion goes through scrubbing and human review, and needs customer consent in the contract.

Make Deletion Reach Every Derived Copy

A customer churns, or a user files a GDPR Article 17 erasure request. Deleting their rows from the conversation table takes an afternoon. Then you find their content in embeddings, in a session summary that also mentions two colleagues, in a semantic cache entry, in 30 days of traces, and in the eval set someone exported in March. The summary is the nasty one, because you can't delete half of a paragraph that an LLM wrote.

Deletion only works if you record lineage at write time. Every memory artifact carries the tenant ID, the user ID where relevant, and the IDs of the source records it was derived from. A summary built from twelve messages lists those twelve. When any source is deleted, every artifact derived from it is deleted or regenerated from what remains. Without that pointer, your only honest option for a deletion request is wiping the whole tenant's derived memory and rebuilding it.

Retention needs a number. Pick a TTL per store, say 90 days for raw conversation memory and shorter for caches, and enforce it with a scheduled job that someone monitors. 'Keep forever because it might help the agent' is a policy too, just one your DPA probably contradicts. For more ideas on treating these stores with database-grade migration and rollback rules, see schema discipline for RAG indexes.

Backups and vendor copies count. If your vector database provider keeps snapshots for 35 days, your deletion SLA is at least 35 days, and your customer contracts should say so. Check whether your LLM provider retains prompts, which carry injected memory, and on what terms.

• →Store source-record IDs on every summary, embedding and fact so deletion can cascade.
• →Set an explicit TTL per memory store and alert when the purge job fails.
• →Include traces, caches, eval exports and vendor snapshots in the deletion runbook.
• →Run a test erasure end to end before launch and confirm nothing for that user is retrievable afterward.

Prove Isolation With Canaries, Not Code Review

You can't verify tenant isolation by reading the retrieval function, because the leak lives in the function nobody thought to read. Test it from the outside instead. Seed each test tenant with unique canary strings, nonsense tokens like a fake project codename that appears nowhere else, written into memory through the normal product path.

Then attack it. From tenant B, run sessions that try to surface A's canaries: direct questions, prompt injections asking the agent to search 'all customers', requests to 'ignore the filter', and ordinary queries semantically close to A's seeded content. Assert in CI that no canary from another tenant shows up in any retrieved context, tool output or final response. Check retrieved context, not just the answer, because a model that sees leaked data and happens not to repeat it is still leaking. Wire this into the same harness you'd use for stress-testing an agent before it ships.

Keep canaries in production too. A handful of synthetic internal tenants with planted strings, plus a monitor that scans outbound responses and retrieval logs for them, gives you a tripwire for regressions that only appear at production scale, like a cache key collision under load.

• →Seed unique canary tokens per test tenant through the real write path.
• →Assert on retrieved context and tool outputs, not only the final answer.
• →Include injection probes that ask for other tenants' data by name.
• →Run synthetic canary tenants in production and alert on any cross-tenant hit.

What's Still Hard, and Where to Start Monday

Some of this is unsolved. Deleting a person from an LLM-written summary that mixes several people means regenerating it, and regeneration can change what the agent remembers for everyone else in that tenant. Semantic caches are hard to partition cleanly because their whole value comes from matching similar inputs, and two tenants asking similar questions is exactly the case you have to block. If anyone has fine-tuned on memory data, deletion becomes a retraining question, and I'd avoid that path entirely for tenant data.

None of that changes the core call. Persistent memory is customer data at rest. It gets storage-level tenant isolation, a tenant scope the model can't override, consolidation that stays inside one partition, lineage so deletion cascades, and canaries that prove all of it. The agent-specific twist is that the model is an untrusted reader, the same framing as treating agents like untrusted insiders.

If you're adding memory this quarter, do the inventory first. List every store that will hold memory or anything derived from it, put a tenant key, an owner and a TTL next to each, and refuse to ship any store where one of those three cells is blank.