Context compression can delete the proof

Compression needs a preservation target

A context compressor takes material that may be too expensive or too long to process directly and produces a smaller representation. The obvious metric is the number of tokens removed. I think the more important question is what must remain invariant after removal. For a casual overview, preserving the main themes may be enough. For a decision about whether a condition is satisfied, the relevant exception or uncertainty may matter more than the dominant theme. Without a task-specific preservation target, a compressor can succeed at reducing text while quietly changing the evidence available to the answering system.

Jiang and colleagues’ LLMLingua studies prompt compression through a budgeted approach and evaluates its effects on selected tasks. That research establishes a useful method and experimental results, not a universal guarantee that compressed context preserves every important relationship. My concern is the evidence contract at the application boundary. The system should know whether the compressed representation is intended as a navigational aid, a sufficient source for an answer or a lossy reminder that requires reopening the original. Those roles permit different losses. Treating them as interchangeable makes an attractive compression ratio look more authoritative than the retained information deserves.

References: [1] LLMLingua — Jiang and colleagues

Correct retained claims can still be incomplete

Consider an illustrative source with ten equally weighted factual claims. A summary retains nine, and every retained claim is accurate. Its retained-claim precision is one, while its coverage of the original claims is nine tenths. That may be acceptable if the missing claim is incidental. It can be disastrous if the missing claim is the only exception to a rule the user is asking about. The arithmetic does not change; the consequence does. I would therefore measure critical-condition coverage separately from a general average, rather than assume a high factual precision score implies sufficient evidence for the task.

Min and colleagues’ FActScore evaluates factual precision using atomic facts and source support. Its distinction between individual factual claims is useful here, but the paper is not a specification for lossless compression. My additional argument is that evaluating only the truth of what remains misses the importance of what disappeared. A compressor can avoid inventing anything and still remove the proof, qualifier or contradiction required for a correct answer. The simple table makes those evaluation dimensions explicit. None of its numbers describe a deployed compressor; they illustrate why compression quality needs more than a single correctness percentage.

References: [2] FActScore — Min and colleagues

Preserve relationships as well as sentences

Evidence often depends on relationships between fragments: a date qualifies a rule, a heading names the population, a table caption defines the units, or a footnote limits the conclusion. Keeping all the nouns while losing those relationships can produce a fluent but misleading summary. I would ask the compressor to preserve claim-condition pairs and explicit uncertainty when the task depends on them. Better still, the source parser can retain structural links so that compression does not have to rediscover every dependency from a flat string. A document’s organization is often part of its meaning, not decorative formatting.

Negation and comparison deserve particular attention because small textual changes can reverse a decision. So do proposal status and temporal scope: planned support is different from implemented support, and a historical policy is different from the current one. A compression evaluation should contain examples where those distinctions determine the answer. The counterargument is that preserving every qualifier defeats compression. That is true if the objective is a universally sufficient replacement for the source. It is less true when the compressor is given a specific question and permission to preserve only the dependencies needed for that question, with a fallback to the original.

Use compressed context as an index when possible

A safer architecture often uses a compact representation to locate relevant source spans, then retrieves the original evidence for the final answer. The summary helps decide where to look without claiming to replace the source. This costs an additional retrieval step and requires stable source identifiers, but it can preserve a much clearer evidence trail. I would attach document revision and span references to compressed claims. If a reference no longer resolves, the system should know that it has a stale pointer rather than silently treating the compressed sentence as an independent and current fact.

This approach is not always feasible. The original may be unavailable during an offline workflow, or the latency budget may forbid another fetch. In that case, the compressed representation needs a stronger acceptance test for its intended use and a clear statement of what it cannot establish. The application can retain a small set of decisive source quotations or structured conditions alongside the summary, subject to its content and privacy requirements. The design question is where to spend the remaining context budget so that the final answer has enough evidence, rather than simply giving every source an equal number of summary tokens.

Repeated summaries compound attribution loss

An agent may summarize a conversation, later summarize that summary and eventually merge it into a durable memory. Each transformation can remove provenance and uncertainty while making the surviving statement sound more settled. I would avoid treating a summary of a summary as equivalent to another independent observation. Its lineage should point back to the same original evidence. If the original is corrected or deleted, derived representations need a policy for invalidation. Otherwise compression becomes a mechanism that launders a weak assumption into a durable fact simply by repeatedly shortening the context around it.

A compression pipeline can retain structured provenance separately from the prose, including source revisions and transformation versions. That metadata consumes resources, but it allows a reviewer to distinguish source facts from generated interpretation. It also helps evaluate whether a regression came from a changed compressor or changed input. I would not promise that this makes every summary reversible; discarded details remain discarded unless the original is retained. The benefit is more modest and useful: the system knows which evidence the summary came from and when it must return to that evidence instead of trusting the compressed representation alone.

Evaluate the whole cost of the shortcut

Compression itself consumes time and computation. A system can spend more generating a carefully shortened prompt than it saves on a single downstream answer. Reuse changes the calculation: compressing a stable source once may pay off across many requests, while compressing every small request may not. I would measure end-to-end latency and total computation, including retries caused by missing context. A token reduction is an intermediate result. The product benefit depends on whether the reduced context improves the complete workload without creating enough rework or answer errors to erase the savings.

LLMLingua’s reported experiments provide evidence for its tested settings, while a deployment needs its own workload-specific comparison. I would compare uncompressed input, a simple extractive baseline and the proposed compressor under the same answer requirements. Include both average cases and examples where the decisive condition is short or easy to overlook. The evaluation should allow abstention when compression leaves insufficient evidence. Forcing an answer on every case can make the compressor appear useful by transferring uncertainty into confident output. A good shortcut should preserve the option to say that the remaining context is not enough.

References: [1] LLMLingua — Jiang and colleagues

Make compression a reviewable transformation

For consequential tasks, I want to inspect a sample of source-to-compressed transformations with the downstream question beside them. That review reveals losses that generic summarization scores can miss. Did the compressor preserve the effective date? Did it retain both sides of a disagreement? Did it turn an uncertain inference into a direct statement? Those checks should inform automated evaluation cases and the format of the compressed representation. A structured record may be more useful than elegant prose when the answer depends on exact conditions. Readability matters, but the preserved decision boundary matters more.

The right compression policy is explicit about what it preserves and what it discards. It can remove repetition, irrelevant detail and redundant framing while retaining the evidence needed for the supported task. When that cannot be guaranteed, it should serve as a pointer to the source or trigger a request for more context. I would rather carry a few additional tokens of qualification than achieve a better ratio by removing the only reason an answer is true. Context is not merely storage to shrink; it is the material from which the system must justify its conclusions.