ADK memory bank — cross-session recall, consolidation, decay, privacy

The boundary: Session and State own the turn, memory owns the relationship

Draw this line once and the rest follows. A Session is one conversation: its ordered events are the raw history the model sees each turn, and its state dictionary is an explicit key-value scratchpad you write and read by exact key. That mechanism is deterministic — set state['plan'] and it is there verbatim until you change it — and the companion article on sessions and state covers it in depth.

Long-term memory is the opposite mechanism: nothing is fetched by key; you ask a question and get back the claims most relevant to it, ranked. Duration does not separate the two — a user:-scoped state key on a persistent SessionService also survives across sessions. The real axis is naming. State holds the few things you can enumerate in advance and must read back exactly: an account id, a running total, a half-filled form. Memory holds everything nobody thought to name — the aside three weeks ago about running Postgres, the preference expressed in passing, the decision and its rationale. If you must never misremember it, it belongs in state; if you could not have predicted needing it, it belongs in the bank.

What is worth persisting

Selectivity at ingest is the cheapest quality lever you have, because every marginal memory permanently taxes every future retrieval. The useful test is durability: will this still be true and useful next month? Facts about the user and their environment pass; the phrasing of today’s request does not.

Two rules keep the bank honest. Never persist a value whose correctness matters exactly — a balance, an order id — because retrieval is fuzzy and a confidently-recalled wrong number is worse than no recall. And treat ‘don’t remember this’ as a hard extraction filter rather than a preference to weigh.

Extraction: turning a finished session into claims

A memory bank stores distilled claims, not transcript chunks. Dumping raw turns into a vector index gives you a store that is bloated and useless at once, and one you can never dedupe, supersede, decay, audit or selectively delete. So extraction is itself an LLM task: a finished session’s events go to a model with a narrow instruction — emit durable claims with a type and a subject; ignore chatter, intermediate reasoning, and anything the user asked to keep private; do not infer beyond what was stated. The output is structured, which is what lets every later stage operate on it.

@dataclass
class Memory:
    text: str            # "prefers TypeScript for new services"
    kind: str            # fact | preference | decision | thread
    subject: str         # "language preference"
    scope: dict          # {"user_id": "u_123", "app": "assistant"}
    confidence: float    # 0.0-1.0, from how firmly it was stated
    created_at: datetime
    source_session: str  # provenance, for audit and deletion
    superseded_by: str | None = None

Two fields earn their keep. Confidence distinguishes ‘I only ever write TypeScript’ from ‘Python’s fine for scripts, I guess’, and becomes a ranking term later. Provenance makes a memory auditable and deletable. The natural trigger is session close, fired from whatever owns the end of a conversation: a UI handler, an idle-timeout sweeper, a nightly batch. Guard it, though — a three-turn ‘what time is it’ exchange yields nothing durable while costing an extraction call and permanent index weight.

session = await session_service.get_session(
    app_name=APP, user_id=user_id, session_id=session_id)

# Not every session earns a memory. Skip the throwaways.
if len(session.events) >= MIN_EVENTS and not session.state.get("temp:no_memory"):
    await memory_service.add_session_to_memory(session)

Sessions that never close need a periodic checkpoint that extracts only events past the last watermark. Either way, keep ingest asynchronous — nobody should wait on the extractor to get a reply.

Scoping: the isolation guarantee

Every memory carries a scope, and every retrieval hard-filters by it. This is the one property of a memory bank that is not a quality concern but a safety one: a cross-user memory leak is not a bug, it is a breach. ADK builds the scope into the interface — search_memory takes app_name and user_id, so a caller cannot accidentally issue an unscoped query — and a managed bank carries the same idea as a scope map attached to every stored memory.

Enforce it at the store layer, not the query layer. A filter applied in application code is one forgotten parameter away from returning everyone’s memories; a per-user namespace, partition, or row-level policy fails closed instead. Broader scopes are legitimate but should be explicit and separate — an org scope for a shared workspace, an app scope for things true of every user — and consolidation must never let a user-scoped memory drift upward into one of them. Make the guarantee testable: an integration test that writes two users’ memories and asserts neither appears in the other’s retrieval belongs in CI, because that is the test whose failure is a security incident.

Consolidation: merge, dedupe, supersede

Without consolidation a bank degrades by accretion. The same preference, restated across ten sessions, becomes ten near-identical vectors that crowd out everything else in the top-k. Consolidation runs on the write path: each newly extracted claim is searched against existing memories in the same scope, and the close matches decide what happens next.

Supersession is what keeps the store truthful. When the user moves from the platform team to infrastructure, the old fact must stop being retrievable as current — but tombstone rather than delete it, because the history matters for audit. Judging ‘is this the same claim?’ is again a model call, and it is where a managed Memory Bank earns most of its price: consolidation is easy to sketch and hard to get right at scale.

Contradiction: when the bank cannot decide

Recency resolves most conflicts, and quietly. It does not resolve all of them. Two statements can be recent, confident, and irreconcilable — ‘text me about incidents’ from Tuesday and ‘email only, please’ from Thursday — or they can be conditionally true in ways neither claim records (‘text me’ was about pages, ‘email only’ about newsletters). Auto-superseding here produces an agent that is confidently wrong half the time.

The honest handling is to keep both, flag the conflict, and let it surface at retrieval. When a search returns two memories the store has marked as contradictory, the agent should see them as a conflict rather than as two independent facts, and its instruction should tell it what to do: ask. ‘I have conflicting notes on how you want to be contacted — which is right?’ beats a coin flip, and the answer resolves the conflict permanently at the next ingest. Watch the rate, too: if unresolved conflicts are common, your extractor is dropping the conditions under which each claim holds rather than your users being fickle.

Retrieval: relevance, recency, and the k budget

Retrieval is where the bank either pays off or embarrasses itself, and vector similarity alone is not enough. Two memories can be equally similar to the query while one is a firm statement from last week and the other a low-confidence aside from a year ago. A memory bank therefore ranks on a blend — similarity, recency, confidence — with superseded entries excluded outright.

def score(mem, sim, now, half_life_days):
    if mem.superseded_by:
        return 0.0
    age_days = (now - mem.created_at).days
    freshness = 0.5 ** (age_days / half_life_days)
    return 0.6 * sim + 0.25 * freshness + 0.15 * mem.confidence

Keep k small. Three sharp memories beat twenty loosely-related ones, because every irrelevant memory is both tokens spent and a chance for the model to anchor on something that did not matter. Apply a relevance floor too: returning nothing is the correct answer when the turn has no history worth recalling. Note the contrast with document grounding, covered in the companion RAG article — there you retrieve authoritative passages from a corpus you control and recall matters most; here you retrieve fallible claims about one person, and precision matters most.

Decay and staleness: giving memories a half-life

Facts age at wildly different rates, and a bank that treats them uniformly will either forget stable truths or confidently assert expired ones. The fix is a per-type half-life applied as a ranking penalty rather than a hard expiry — an old memory should lose to a fresh one, not vanish, because sometimes the old one is all you have.

Open threads are worth special-casing: ‘waiting on the security review’ is useful for two weeks and actively misleading after three months, so give it a TTL and let expiry either drop it or turn it into a question. Decay also buys cheap compaction — memories below a decay threshold that have never once been retrieved are the safe candidates for pruning.

Grounding memories into context, with provenance

How a retrieved memory enters the prompt determines how much damage a wrong one does. Injected as bare assertions, memories launder into ground truth: the model treats ‘the user is on the platform team’ as fact and reasons confidently from it six months after they moved. Injected with attribution and a date, the same memory becomes evidence the model can weigh and, if the conversation contradicts it, override.

What you remember about this user (may be outdated; the
current conversation always wins):
- [2026-05-02, high confidence] Prefers TypeScript for new services.
- [2026-06-18, medium] Migrating a legacy Python service off Heroku.
- [conflict] Contact preference: "text me" (07-14) vs "email only" (07-21).
  Ask the user which applies before acting.

Three rules make this block safe. Cap it — a fixed budget of memories and characters, so a growing bank cannot slowly eat the context window. Date everything, so the model can discount age itself. And state the precedence rule explicitly: what the user says now outranks anything the bank remembers. That last line turns memory from a source of stubborn, confidently-wrong behaviour into a prior the conversation may update.

Privacy: consent, deletion, and audit

A session is ephemeral; a memory bank is a durable, growing dossier of personal facts, and it attracts the full weight of data governance. Three obligations are architectural — you cannot bolt them on after the store is full.

Consent is an ingest-time gate: the user should know memory is on, be able to turn it off, and be able to say ‘don’t remember that’ about a specific exchange — which means the extractor needs a suppression path, not just a policy page. Deletion must actually delete: purge the vectors from the index rather than tombstone the rows, then verify a retrieval no longer surfaces them. Because memories are per-user scoped and carry their source session, both ‘forget everything about me’ and ‘forget that one conversation’ are executable queries rather than research projects. Audit closes the loop: the user should be able to see what the agent believes about them and where each belief came from. That view is also the best debugging tool you will have — most ‘why did the agent say that?’ incidents resolve the moment you read the memories it retrieved.

Managed Memory Bank versus building your own

ADK lets you plug either in behind the same interface, so this is a deployment decision rather than a rewrite. VertexAiMemoryBankService gives you the managed product: extraction, consolidation and scoped semantic retrieval operated for you. VertexAiRagMemoryService indexes ingested sessions into a RAG corpus — semantic retrieval without the consolidation layer. Or implement BaseMemoryService over a vector store you already run.

The honest default is managed. The hard parts of a memory bank are the ones you cannot see from outside — extraction quality, consolidation judgement, supersession — and they are exactly what you would have to build and keep evaluating. Build your own when you need a domain-specific schema, when residency or retention policy forbids the managed store, or when memories must live in a database you already query for other reasons.

Evaluating a memory bank

Memory failures are silent: an agent that failed to recall looks merely unhelpful, and one that recalled something stale looks confident. Build a small labelled set of multi-session scenarios — the memories that should exist after ingest, and the ones that should surface on a later turn — and track four numbers. Extraction precision and recall: did ingest capture the durable claims and skip the noise? Retrieval precision at k: of the memories surfaced, how many belonged there. Staleness rate: how often a superseded or expired fact surfaces as current, which catches broken consolidation faster than anything else. And the leak test, pass/fail rather than a metric. Re-run all four on every change to the extraction prompt or the ranking weights — both are exactly the kind of tuning that regresses without anyone noticing.