Glonce

Naive RAG baseline Raw-RAG beats agentic memory systems on long-conversation benchmarks

researchThe Glonce editorsHugging Face🔥58Glonce rating58Impact42Novelty52Relevance98Surprise45Players8Credibility75
Naive RAG baseline Raw-RAG beats agentic memory systems on long-conversation benchmarks

A new paper asks whether the complexity of agentic memory for LLMs is always necessary. Such systems, the authors note, often rely on mutation, structural organization, learned retrieval strategies or other complex operations to support long-term conversation understanding.

The authors study Raw-RAG, a deliberately naive RAG baseline. It uses dense retrieval and reranking over the raw conversation history to supply evidence to a reader LLM, with no transformation of the stored memory.

They test it across four benchmarks: LoCoMo, LongMemEval, MEME and MemoryAgentBench. Raw-RAG ranks first among the compared memory systems in six of seven benchmark-model settings. It outperforms the strongest compared agentic memory system by 0.13-5.53 points.

The paper then investigates why raw-history retrieval stays competitive. Its evidence analysis finds that Raw-RAG's higher coverage of the necessary evidence can offset its lower utilization of that evidence. Useful evidence, the authors say, includes both the answer values and the relations and dependency paths needed to derive them. They also report that even complete access to transformed memory stores leaves an accuracy gap relative to raw evidence, which they take as a sign of how much preserving raw information matters.

Their conclusion is scoped: the findings challenge the necessity of complex agentic memory for the evaluated conversation tasks, and establish naive RAG as an essential baseline for this kind of work.

Key facts

  • Raw-RAG is a naive baseline: dense retrieval plus reranking over raw conversation history, feeding a reader LLM.
  • It ranks first among the compared memory systems in six of seven benchmark-model settings across LoCoMo, LongMemEval, MEME and MemoryAgentBench.
  • It beats the strongest compared agentic memory system by 0.13-5.53 points.
  • The authors say higher coverage of necessary evidence can offset lower utilization.
  • Even full access to transformed memory stores leaves an accuracy gap versus raw evidence, so preserving raw information matters.

Why it matters

Agentic memory systems add mutation, structural organization or learned retrieval on top of conversation history. This paper argues that a plain retrieve-and-rerank pipeline over the raw history is often just as good or better on the evaluated tasks. If that holds up, it sets a stronger default for anyone designing LLM memory: beat naive RAG first, then justify the added machinery.

Who it affects

Researchers building and benchmarking memory for long-term conversation, and engineers deciding how much memory infrastructure an assistant needs. The authors say the result establishes naive RAG as an essential baseline.

How to use it

The practical takeaway is to include a raw-history RAG baseline (dense retrieval plus reranking feeding a reader LLM) in any comparison of memory systems, and to keep raw information available rather than relying only on transformed memory stores. The source text does not mention released code or datasets.

How solid is it

The claim rests on four named benchmarks and a win in six of seven benchmark-model settings, with margins of 0.13-5.53 points. Some of those margins are small. The text does not say which metric the points measure, which memory systems were compared, or which setting Raw-RAG did not win. The authors scope their own conclusion to the evaluated conversation tasks.

Risks and caveats

The result is a preprint and covers only conversation-memory benchmarks, so it does not show that complex memory is unnecessary elsewhere. Raw-RAG's win comes from covering more of the needed evidence despite using it less well, which may not carry over to other settings. No cost, latency or token-usage comparison is given.