LycheeMemory V2 cuts memory-construction tokens by up to 86%

Long-horizon LLM agents need to retain information from past interactions to handle future tasks, but most memory systems do this through eager consolidation: an LLM is invoked after every single interaction to extract, summarize or update the agent's memory. As conversations grow longer, this turn-by-turn approach makes memory construction increasingly expensive. The paper argues that the usual workarounds carry their own costs: coarser summarization lowers construction cost but risks losing fine-grained contextual evidence, while larger retrieval contexts or multi-hop LLM reasoning at query time simply move the expense from construction to query.

LycheeMemory V2 replaces turn-level consolidation with what its authors call semantic segment-level consolidation. Rather than invoking an LLM after each exchange, the system batches multiple exchanges into segments and, once a segment is finalized, encodes it into context-independent typed memory records; this segment-level batching lowers how often an LLM has to be invoked for encoding. To decide where a segment ends, LycheeMemory V2 uses semantic boundary detection rather than a fixed window size, which the authors say helps preserve coherent event-level and temporal evidence compared with fixed-window batching. The resulting records sit behind lightweight structured indexes built for query-planned evidence retrieval.

Tested with GPT-4.1-Mini, LycheeMemory V2 reaches what the authors call state-of-the-art performance on two long-term memory benchmarks: 89.22% on LoCoMo and 92.20% on LongMemEval-S. Measured against A-Mem, an existing memory system used here as the comparison baseline, LycheeMemory V2 cuts the tokens spent on memory construction by 86.0% on LoCoMo and by 75.9% on LongMemEval-S, and it does so without increasing the number of tokens spent at query time.

The authors draw a broader conclusion from this: for long-term agent memory, the trade-off between accuracy and cost is shaped not only by what information a system chooses to retain, but by the granularity at which that information gets consolidated in the first place.

Key facts

LycheeMemory V2 replaces turn-level memory consolidation with semantic segment-level consolidation, batching multiple exchanges into segments before encoding each one into memory records.
The system decides segment boundaries with semantic boundary detection rather than a fixed window, which the authors say better preserves event-level and temporal evidence.
Tested with GPT-4.1-Mini, LycheeMemory V2 reaches 89.22% on the LoCoMo benchmark and 92.20% on LongMemEval-S, state-of-the-art scores according to the authors.
Compared with A-Mem, LycheeMemory V2 cuts memory-construction tokens by 86.0% on LoCoMo and 75.9% on LongMemEval-S, without increasing query-time token usage.
The authors conclude that the accuracy-cost trade-off in long-term agent memory depends not only on what is retained, but also on the granularity at which it is consolidated.

Why it matters

As LLM agents run longer conversations across many sessions, giving them memory of what happened before becomes necessary, but most current memory systems consolidate memory after every single interaction, invoking an LLM each time. That per-turn cost compounds as a conversation grows, and it is the exact bottleneck LycheeMemory V2 targets. By consolidating in batched segments instead of on every turn, the system lowers how often an LLM needs to be invoked for encoding, and the authors report this comes without sacrificing accuracy or adding cost at query time, the two most obvious ways a cheaper memory system could quietly get worse.

Who it affects

The paper is aimed at teams building LLM agents that hold long conversations or persist across many sessions, such as assistants or coding agents that need to recall earlier context without re-reading an entire transcript. Because construction tokens are a direct, recurring cost of running such an agent, the reduction matters most to teams operating agent memory at scale rather than in a one-off demo. It is also relevant to researchers working on the same benchmarks, LoCoMo and LongMemEval-S, and to anyone currently using A-Mem or a comparable memory framework.

How to use it

LycheeMemory V2 is introduced here as a method inside a research paper. For a team building an agent memory pipeline, the practical takeaway is architectural: replace turn-by-turn consolidation with segment-level consolidation, batching exchanges and waiting for a semantic boundary before encoding a segment into typed memory records, served through a lightweight structured index at query time. Anyone benchmarking their own long-term memory system can use the same yardsticks reported here, the LoCoMo and LongMemEval-S benchmarks, with A-Mem as the construction-token baseline and GPT-4.1-Mini as the model the authors tested with.

How solid is it

The headline numbers come from experiments on two established long-term memory benchmarks, LoCoMo and LongMemEval-S, using GPT-4.1-Mini as the underlying model, with A-Mem serving as the named comparison system for the token-reduction figures. That gives the token savings a concrete point of reference: an existing system tested the same way. The accuracy figures are a different matter: the paper reports 89.22% and 92.20% and calls them state-of-the-art, but it does not name a prior best score or a baseline number for those two figures the way it does for the token counts, so this text alone does not show how large the accuracy gap over the next-best system actually is. The results also come from a single model, GPT-4.1-Mini; nothing here indicates whether the same gains would hold with a different LLM.

Risks and caveats

The abstract does not explain mechanically what 'typed memory records' or 'semantic boundary detection' consist of, only that the system uses them, so it is not possible from this text alone to judge how the segmentation is implemented or how reproducible it is. No dataset sizes, latency numbers, dollar costs or hardware details are given, so the real-world speed and cost of running LycheeMemory V2, beyond the relative token counts, are not established here.

“Segment-level batching lowers LLM encoding frequency, while semantic boundary detection helps preserve coherent event-level and temporal evidence compared with fixed-window batching.”

— the paper's abstract