Agent memory needs forgetting

Memory is a record with a lifecycle

A durable agent memory is not the same thing as a model’s attention cache. It is application data retained across interactions: preferences, task state, source facts or summaries intended to influence future behavior. That makes it a storage and governance problem as well as a retrieval problem. I would ask why each record exists, where it came from and when it should stop being used. A system that stores every inferred fact indefinitely may appear helpful at first while gradually accumulating stale assumptions that the user never meant to turn into permanent instructions.

Packer and colleagues’ MemGPT explores management of limited model context using a hierarchy of memory resources. That is useful background for separating working context from external storage, but it does not remove the application’s responsibility to define retention and correction. My concern here is the lifecycle of the stored record, not the mechanics of paging it into a prompt. A memory can be efficiently retrieved and still be wrong, obsolete or inappropriate for the current task. The quality of the storage policy therefore places a limit on the quality of any retrieval strategy built above it.

References: [1] MemGPT: Towards LLMs as Operating Systems — Packer and colleagues

Separate statements from inferred preferences

A direct user statement, an observed action and a model’s inference should not become indistinguishable memory entries. If someone chooses a short answer once, that is evidence about one interaction, not necessarily a permanent preference for brevity. I would store the origin and strength of the claim, along with the scope in which it applies. The system can then use explicit preferences more confidently and treat inferred ones as revisable suggestions. Flattening both into an unconditional instruction gives an uncertain inference more authority every time it is repeated in a future context.

Provenance helps keep those distinctions intact. The W3C PROV data model describes entities, activities, agents and derivation relationships that can inform a memory record’s lineage. I would use only as much structure as the application needs, but preserve the essential answers: which source supported this record, which transformation produced it and which revision was involved? A concise summary can still point back to its source. Without that link, a later correction becomes difficult because the system cannot distinguish an independent fact from a restatement of the same mistaken inference stored in several places.

References: [2] W3C PROV Data Model

Expiry is different from contradiction

Some memories become less useful with time even when they were accurate when recorded. A temporary travel preference, a project deadline or a current role may need an expiry or a revalidation rule. Other memories are actively contradicted by a newer statement. Those conditions should be represented separately. Expiry says the record no longer has sufficient freshness for automatic use; contradiction says there is evidence that its content should change. I would avoid silently choosing whichever record has the highest embedding similarity, because semantic resemblance does not establish which version is current or authoritative.

A practical policy can distinguish active, needs-review, superseded and deleted records. These are application states, not assertions that the model has forgotten information in its parameters. The retrieval layer should filter by the relevant state before assembling context, and the answer generator should receive uncertainty when a current conflict remains unresolved. Automatic expiration is useful for clearly temporary data, but arbitrary short lifetimes can erase valuable continuity. The counterargument that forgetting reduces personalization is valid. The response is to choose retention based on purpose and user control, not to assume that either permanent storage or aggressive deletion is universally correct.

Deletion must follow derived records

Suppose an original note produces a summary, and that summary contributes to a profile entry. Deleting only the original leaves its information available through the derived records. A memory system therefore needs a dependency policy. It may delete derived records, invalidate them for recomputation or remove only the affected claims if their provenance is sufficiently precise. The correct choice depends on how the derivation is represented. A broad summary with no claim-level lineage may require broad invalidation. That is a real cost of lossy summarization, not a problem that can be solved by deleting one row and declaring the job complete.

The small graph example below computes a transitive invalidation set from a removed source. It uses synthetic identifiers and demonstrates reachability only. A production deletion process also has indexes, replicas, caches, exports and backup retention to consider, with completion semantics appropriate to the system. I would record which stores have stopped serving the data and which retention processes remain pending. This is an engineering consistency requirement, not a claim about any particular legal obligation. The user-facing description should accurately distinguish immediate retrieval exclusion from physical removal across every storage layer.

from collections import deque
derived = {'source': ['summary'], 'summary': ['profile'], 'profile': []}
def affected(start):
    queue, seen = deque([start]), set()
    while queue:
        item = queue.popleft()
        if item in seen:
            continue
        seen.add(item)
        queue.extend(derived.get(item, []))
    return seen
assert affected('source') == {'source', 'summary', 'profile'}
assert affected('summary') == {'summary', 'profile'}

Prevent deleted information from reappearing

An asynchronous summarizer can race with a deletion request. It may read an old source, finish after the source was removed and write a fresh derived memory that reintroduces the information. The write path needs to check source state or revision at commit, not only when the job starts. Tombstones or generation markers can help reject stale work. The same principle applies to index rebuilding: a rebuild from an outdated snapshot must not make a deleted record searchable again. Deletion is a protocol across readers and writers, not just a command against the current primary database.

There is also a distinction between removing a stored memory and preventing the system from inferring a similar fact again from new, legitimately available evidence. The product should explain what its forget operation means. It might delete existing records, suppress future storage of a category or both. Those are different policies and need different controls. I would avoid promising that an application has erased all possible knowledge of a fact when it only removed one memory entry. A precise promise is more useful: identify the retained data under application control, define how it is excluded or removed and honor that definition consistently.

Retrieval should spend a relevance budget

Even valid memories can become clutter. Injecting every historical preference into every prompt consumes context and can distort the current task. I would retrieve memory according to the task’s needs and distinguish instructions from background facts. A preference relevant to code formatting may be irrelevant to a research summary. A saved project fact may need fresh verification before being used in a current recommendation. The memory store should make those distinctions available rather than force the generator to infer them from a pile of unstructured prose whose age and authority are invisible.

A useful evaluation asks whether memory improves continuity without overriding the current user request or introducing stale assumptions. Include explicit corrections, temporary preferences, unrelated tasks and requests to forget. Measure false persistence as well as successful recall. A system that remembers nine helpful details but repeatedly revives one corrected personal assumption may feel less trustworthy than a simpler assistant with less memory. MemGPT’s context-management perspective helps explain why information must be selected for working context; my additional argument is that the selection policy should consider lifecycle and authority alongside semantic relevance and token cost.

References: [1] MemGPT: Towards LLMs as Operating Systems — Packer and colleagues

Make memory inspectable enough to correct

Users need a practical way to see and change consequential stored assumptions. That does not require exposing internal embedding vectors or every low-level event. It does require presenting the meaningful records, their source or explanation, and the controls that affect future use. A correction should update the authoritative record and trigger the necessary derived-data handling. If the interface edits a display label while the old summary remains in the retrieval index, the control is misleading. I would test the complete path from a user correction to the next answer that could have used the previous value.

The memory I want is selective, attributable and revisable. It retains enough context to make an assistant useful without treating every past interaction as permanent truth. Forgetting is part of that design: expiration handles changing relevance, correction handles changing belief, and deletion handles retention choices. Keeping those operations distinct makes both the implementation and the user promise clearer. A durable memory system should earn its influence on future decisions through provenance and lifecycle rules, rather than accumulate authority simply because a sentence happened to be written into a database once.