Microsoft Research's EvoLib turns AI agent experience into reusable knowledge
Microsoft Research introduced EvoLib, a framework described in the paper "Test-Time Learning with an Evolving Library," that lets AI agents learn from their own experience while running, rather than just accumulating a growing archive of past conversations and action histories. The team's starting problem: memory alone is not learning. A pile of past reasoning traces makes it harder, not easier, to find the knowledge relevant to a new task. EvoLib instead turns each past attempt into one of two units: a reusable skill distilled from a successful solution, or a reflective insight learned from a mistake. Two mechanisms keep that library evolving. Consolidation retrieves similar existing knowledge whenever a new unit is extracted and merges it into a more general, more reusable version, so knowledge moves beyond a single episode and starts applying across tasks. A weighting mechanism continually re-scores each knowledge unit, based both on how useful it was for the task at hand and on how much it helps generate useful knowledge later, so the entries with the biggest long-term payoff rise to the top of the library over time. Because EvoLib works entirely outside the model, it needs no fine-tuning or weight updates, and Microsoft Research says it can be applied to any black-box language model or AI system accessed only through an API. The team tested it on three kinds of tasks: mathematical reasoning problems, writing code under efficiency constraints, and making decisions in long-horizon, exploratory environments. Across all three, EvoLib beat the strongest retrieval-based memory systems and other abstract-memory approaches the team compared it against, while using fewer tokens. Plotting performance against the amount of test-time compute spent, EvoLib beat both isolated compute-scaling methods and memory-based learning baselines across most of the compute range, and its performance climbed faster as compute increased. A separate test checked whether the approach depends on the order tasks arrive in: running the same set of heterogeneous tasks in different orders, EvoLib still outperformed existing memory-based methods and held stable performance regardless of ordering, which the team says points to it working on a mixed, unstructured stream of real-world requests. Code and experiment results are posted on GitHub for further research on memory and knowledge evolution in AI systems.
Key facts
- EvoLib lets language models learn from their own experience at inference time, with no ground-truth labels, no external feedback, and no model weight updates
- Knowledge units take two forms: reusable skills distilled from successes and reflective insights drawn from mistakes
- A consolidation mechanism merges new knowledge with similar existing entries into more general, reusable units; a weighting mechanism reweights entries by both immediate and long-term usefulness
- Tested on math reasoning, code generation under efficiency constraints, and long-horizon decision-making tasks, EvoLib outperformed retrieval-based and other abstract memory approaches while using fewer tokens
- EvoLib held stable performance across different orderings of the same heterogeneous task set, and code plus experiment results are published on GitHub
Why it matters
AI agents increasingly rely on memory, storing and retrieving past conversations, reasoning traces, and action histories, but a growing archive is not the same as learning: the more experience piles up, the harder it becomes to find what is actually relevant to a new task. EvoLib's premise is that useful learning means distilling experience into a small set of general, reusable lessons, the way people remember strategies and mistakes rather than every detail of what happened, and that this distillation has to keep refining itself as new experience arrives rather than just growing a bigger pile.
Who it affects
The framework targets teams building AI agents and AI systems on top of language models, particularly ones that only have API access to a black-box model and cannot fine-tune or update its weights. Because EvoLib operates entirely outside the model, it is aimed at exactly that deployment pattern: production agents that need to keep improving after they ship without retraining anything.
How to use it
EvoLib does not require access to model internals or any weight updates, so it can be layered onto any black-box LLM or AI system reached through an API. Microsoft Research has published code and experiment results on GitHub, aimed at supporting further research on memory and knowledge evolution in AI systems; the blog post does not describe a packaged product or a pricing or licensing model.
How solid is it
Microsoft Research evaluated EvoLib on three distinct task types, mathematical reasoning, code writing under efficiency constraints, and decision-making in long-horizon exploratory environments, and reports it consistently outperforming both top retrieval-based memory approaches and other abstract memory mechanisms while using tokens more efficiently. Plotted against test-time compute budget, EvoLib beat both compute-scaling baselines run on isolated tasks and memory-based learning approaches across most of the compute range tested, with faster performance gains as compute increased. A separate robustness check ran the same heterogeneous task set in different orders and found EvoLib's performance stayed stable and ahead of existing memory-based methods regardless of ordering.
Risks and caveats
The results come from Microsoft Research's own paper and blog post; the specific benchmarks, baseline systems, and numeric performance gaps are not named or quantified in the text, so the size of the improvement and how it holds up outside these three task categories is not established here. No independent replication is mentioned, and the post gives no timeline or plan for turning EvoLib into a supported product beyond the GitHub research release.
“Memory has become an important AI agent capability: the ability to store and retrieve past experiences. But memory alone is not learning.”
— Microsoft Research, EvoLib blog post