Glonce

Memorilla packs document collections into 16 memory tokens for frozen LLMs

researchThe Glonce editorsHacker News🔥65Glonce rating65Impact50Novelty65Relevance100Surprise40Players10Credibility75

A machine learning paper on arXiv (first submitted 15 October 2026, last revised 3 November 2026 as v3) introduces Memorilla, a memory system for large language models whose weights stay frozen.

The problem the paper starts from: memory lets an LLM use information beyond its parameters, such as past user conversations, long documents or an agent's earlier actions. For a frozen model, that knowledge reaches it only through what a memory system places in the context window. Gathering and combining the information relevant to each query while keeping the context short remains challenging for existing systems, the abstract says.

Memorilla's answer is to gather information from an entire document collection into a fixed number of query-dependent "memory tokens". Documents, such as user chats or book chapters, are encoded once into reusable embeddings. The system then learns to retrieve query-specific information and place it in the LLM's input embedding space.

The headline results: using only 16 memory tokens, Memorilla achieves the best performance against baselines including Mem0 and MemoRAG on five of eight benchmarks. The benchmarks span personalization, long-document and knowledge-intensive question answering. On FactKG, it reaches 84.33% accuracy versus 80.43% with full-context inference, while using 108x fewer tokens to represent the source documents.

The paper also tests an agentic setting, where the number of documents (the agent's past turns) grows throughout the trajectory. The authors say Memorilla can facilitate continual learning there. In a TextWorld environment, it reduces the agent's tendency to wander between rooms it has already explored, and lifts the average success rate from 18.6% with full-history in-context learning to 24.2%.

The abstract's closing claim is that Memorilla helps frozen LLMs accumulate and use knowledge across long-context interactions, and takes a step towards continually learning AI systems.

Key facts

  • Memorilla compresses a whole document collection into a fixed set of 16 query-dependent memory tokens for a frozen LLM.
  • It reports the best performance against baselines including Mem0 and MemoRAG on five of eight benchmarks covering personalization, long-document and knowledge-intensive QA.
  • On FactKG it scores 84.33% accuracy versus 80.43% with full-context inference, using 108x fewer tokens to represent the source documents.
  • In a TextWorld environment, average success rate rises from 18.6% (full-history in-context learning) to 24.2%, and the agent wanders less between explored rooms.
  • Documents are encoded once into reusable embeddings; the system learns to retrieve and place query-specific information in the LLM's input embedding space.

Why it matters

Frozen LLMs can only use what a memory system puts in their context window, and long context is costly. Memorilla takes a different route from stuffing retrieved text into the prompt: it places query-specific information directly in the model's input embedding space as a fixed 16 tokens. The FactKG result is the striking one: higher accuracy than full-context inference (84.33% versus 80.43%) with 108x fewer tokens representing the source documents. The agent experiment points at memory for systems whose history keeps growing.

Who it affects

Teams building memory layers for chat assistants, personalization features and long-document question answering, who today compare systems such as Mem0 and MemoRAG. Developers of agents whose past turns accumulate over a run are the other audience, since the paper tests exactly that case in TextWorld.

How to use it

This is a research paper, and the source gives no installation steps or usage instructions. The described design is the thing to take away: encode documents once into reusable embeddings, then at query time learn to retrieve and place the relevant information into the frozen LLM's input embedding space as a fixed number of memory tokens. The abstract does not say whether code or weights are released.

How solid is it

The claims come from the paper's own abstract on arXiv, which is at version 3 (v1 on 15 October 2026, v2 on 27 October, v3 on 3 November). The abstract does not say whether the results are peer-reviewed or whether code or weights are released. The best-performance claim covers five of eight benchmarks, so Memorilla did not lead on all of them. The abstract does not name the other three of the eight benchmarks, nor the five on which Memorilla was best, apart from FactKG. The TextWorld success rates are averages as the abstract states; the number of runs or variance is not given.

Risks and caveats

The 108x token reduction is given only for FactKG; the abstract does not say it holds on the other benchmarks. The abstract does not say which frozen LLM(s) were used. No details are given on training cost, latency or compute overhead of Memorilla, which matters because the system learns a retrieval and placement step. The TextWorld gain is modest in absolute terms: 24.2% versus 18.6% average success.