JitMem defers agent memory curation from write time to read time
Researchers propose Just-in-Time Memory (JitMem), a new design for agentic memory systems, the mechanisms that let large language model agents reuse past experience to perform better on future tasks. Most existing memory systems curate at write time: once a task finishes, its trajectory is distilled into a fixed artifact, such as a reflection, workflow, skill, or reasoning strategy, which is later retrieved by similarity when a new task arrives. The paper argues this forces the system to decide what is worth remembering before the future query is even known, irreversibly discarding information and producing a query-independent summary that has to serve many possible downstream tasks. Learning a good write-time curator is also hard, the authors say, because the value of a storage decision may only become apparent when a relevant query arrives, sometimes many tasks later, creating a long-horizon credit-assignment problem. JitMem instead keeps raw trajectories in memory and defers curation until read time, once the current task is known: given the retrieved traces and the new task, a memory curator synthesizes a compact, task-adaptive payload tailored to the immediate need. Because this payload is consumed on the same task it was built for, the curator can be trained directly from immediate task success, sidestepping delayed utility signals and the need to artificially group related tasks. The authors test JitMem across three agent benchmarks: ALFWorld, WebShop, and tau^2-bench. JitMem consistently outperforms both no-memory agents and heuristic or learned write-time memory methods, beating the strongest baseline by 16.2 absolute success-rate points on ALFWorld, 16.3 points on WebShop, and 3.9 points on tau^2-bench. Notably, even an untrained curator is already competitive with or surpasses these baselines, which the authors take as evidence that task-adaptive read-time curation itself, not just the training, is a major source of the gain; training the curator on top of that further compounds the improvement.
Key facts
- JitMem defers memory curation from write time (right after a task ends) to read time (once the new task is known), rather than distilling a fixed, query-independent artifact in advance
- It keeps raw trajectories and synthesizes a compact, task-adaptive payload from retrieved traces only when a new task arrives
- Because the payload is consumed on the same task it was built for, the curator can be trained directly from immediate task success, avoiding delayed credit-assignment problems
- Tested on ALFWorld, WebShop, and tau^2-bench, JitMem beats the strongest write-time baseline by 16.2, 16.3, and 3.9 absolute success-rate points respectively
- Even an untrained curator is already competitive with or beats the baselines, suggesting the task-adaptive read-time approach itself drives much of the gain, with training compounding it further
Why it matters
Most agent memory systems commit to a summary of a completed task before knowing what future task will need it, which the authors say irreversibly throws away information and forces one fixed artifact to serve many possible later queries. JitMem's read-time approach only decides what matters once the actual next task is known, which the results suggest closes a real performance gap rather than a marginal one.
Who it affects
The work targets builders and researchers of LLM agents that need to reuse past experience across tasks, in domains represented by the three benchmarks used: household task simulation (ALFWorld), online shopping (WebShop), and dialogue-driven task completion (tau^2-bench).
How to use it
JitMem is presented as a research method rather than a released product: raw trajectories are stored, and at read time a curator model synthesizes a task-specific memory payload from the traces retrieved for the current task. The curator can be trained directly from whether the task succeeds, without needing hand-grouped related tasks or delayed reward signals, though the paper does not spell out the training procedure or model architecture in detail.
How solid is it
The account here rests on the paper's own abstract on Hugging Face Papers, which reports only the relative improvement over the strongest baseline (16.2, 16.3, and 3.9 points) on the three benchmarks rather than absolute success rates for JitMem or the baselines, and does not name authors, institutions, or a publication date.
Risks and caveats
The gains are measured against the strongest baseline on three specific benchmarks and are reported as point differences rather than absolute scores, so the underlying difficulty and ceiling of each task are not directly visible. The paper also does not describe the curator's training method or architecture, leaving open how the approach would generalize beyond the tested settings.
“This forces the system to decide what is worth remembering before the future query is known, irreversibly discarding information and producing a query-independent summary that must serve many possible downstream tasks.”
— from the paper's abstract