Spatial Memory Agent improves frozen VLM spatial reasoning without retraining

Spatial intelligence, judging distance, layout and 3D structure, is described in the paper as a foundation for embodied agents, robotic planning and multimodal assistants. According to the authors, existing work on improving the spatial reasoning of vision-language-model (VLM) agents has mainly followed two lines: post-training methods such as supervised fine-tuning and reinforcement learning, or an agentic paradigm in which the model calls external spatial tools, such as depth estimation and 3D reconstruction tools, to gather intermediate spatial evidence. The paper sets out to test a third, underexplored route: can a frozen VLM agent improve its spatial reasoning through parameter-update-free self-evolution, without depending on external expert spatial tools at inference time?

Their answer is Spatial Memory Agent (SMA), an experience-grounded runtime framework that converts verified spatial experience into reusable, transferable lessons. In a verifiable spatial environment, SMA queries the frozen VLM, obtains a predicted answer and a reward, then uses verifier-guided reflection to distill compact lessons from that experience. Each lesson gets a Transfer Reliability Score (TRS): initialized uniformly at first, then calibrated over time from later retrieval outcomes, which serve as evidence of how reliably that lesson transfers. During read-only deployment, SMA retrieves lessons through a semantic filter combined with similarity-TRS ranking, and that retrieved memory guides the frozen model's inference.

The authors evaluated SMA across five representative spatial benchmarks and four base VLMs, twenty benchmark-model combinations in total. SMA achieves the highest macro average in every one of the four base-model blocks, and the best accuracy among the evaluated methods in most, though not all, of the 20 evaluations. The paper does not name the five benchmarks or the four base VLMs, nor does it give accuracy figures, percentage points or score margins for any of these results, and it reports no numeric comparison against the two existing lines of work, post-training and tool-augmented agentic methods, that it positions itself against. On this basis, the authors describe SMA as establishing a practical parameter-update-free path for spatial self-evolution across the model scales and environments they tested.

Key facts

Spatial Memory Agent (SMA) is a parameter-update-free, experience-grounded framework that helps a frozen VLM agent improve its spatial reasoning without calling external spatial tools such as depth estimation or 3D reconstruction.
SMA builds lessons through verifier-guided reflection: in a verifiable spatial environment it queries the frozen VLM, gets a predicted answer and a reward, then distills that experience into compact, transferable lessons.
Each lesson carries a Transfer Reliability Score (TRS), initialized uniformly and then calibrated from later retrieval outcomes, which SMA combines with similarity to rank the lessons it retrieves during read-only deployment.
Across five spatial benchmarks and four base VLMs, twenty evaluations in total, SMA achieves the highest macro average for every base model.
SMA also gets the best accuracy among the evaluated methods in most, but not all, of the 20 evaluations; the paper names no specific benchmarks, base VLMs, or accuracy figures.

Why it matters

Spatial intelligence, judging distance, layout and 3D structure, is described in the paper as a foundation for embodied agents, robotic planning and multimodal assistants. Existing work on improving it in VLM agents mostly falls into one of two camps: post-training methods such as supervised fine-tuning and reinforcement learning, which update the model's parameters, or an agentic setup where the model calls external spatial tools, such as depth estimation and 3D reconstruction, to gather evidence while it runs. SMA tests a third route: whether a frozen VLM, one whose parameters never change, can still improve at spatial reasoning purely by accumulating and reusing its own verified experience as lessons, without retraining and without external tools at inference time. That is the paper's own framing of why the question is worth asking: self-improvement that needs neither retraining a model nor wiring it up to external tool infrastructure.

Who it affects

The paper does not name a company, product, or specific model family. Its own framing points to embodied agents, robotic planning, and multimodal assistants, the application areas it opens with, as the domains where spatial reasoning is described as foundational. In practice, that puts researchers and engineers building or evaluating VLM-based agents in those domains as the direct audience: SMA is aimed at teams that want to improve a frozen VLM agent's spatial reasoning without retraining it or adding external spatial tools such as depth estimation or 3D reconstruction at inference time.

How to use it

There is no product, price or license to weigh here: this is a research paper, and it does not mention any code, model weights or dataset release. What it does specify is the mechanism. Lesson-building happens in a verifiable spatial environment, where SMA queries the frozen VLM, checks the predicted answer against a reward, and uses verifier-guided reflection to write a compact lesson tagged with a Transfer Reliability Score that starts uniform and is calibrated as the lesson gets reused. At deployment, that stage is read-only: the frozen model's parameters are not touched, and SMA retrieves relevant lessons through a semantic filter combined with similarity-TRS ranking, then feeds them in to guide the model's inference. With no public code or dataset named in the source, this describes how SMA is designed to work rather than something a reader can install today.

How solid is it

The evaluation is reasonably broad on its face: five spatial benchmarks and four base VLMs, 20 benchmark-model combinations in total, with SMA reported as reaching the highest macro average in every one of the four base-model blocks. But the paper does not name any of the five benchmarks or four base VLMs, and it gives no accuracy numbers, percentage points or score margins for the macro-average or best-accuracy results, so the size of SMA's advantage cannot be judged from the source text. The paper is also explicit that SMA does not sweep every evaluation: it gets the best accuracy in most, not all, of the 20, though it does not say in how many cases it fell short or against which method. No numeric comparison is given against the two existing lines of work, post-training methods and tool-augmented agentic methods, that the paper positions SMA against. No author names, institutional affiliations or publication venue appear in the text either.

Risks and caveats

The headline claims come from the paper's own abstract, with no independent benchmark numbers, named baselines or named benchmarks to check them against, so readers cannot tell how large SMA's macro-average lead actually is, or how close its win in most of the 20 evaluations comes to a clean sweep. The Transfer Reliability Score, the mechanism meant to keep unreliable lessons from being retrieved, is described only in terms of how it is calibrated, not with any measurement of how well it filters bad lessons in practice. No code, model weights or dataset are mentioned as available, so the results are not independently reproducible from the source alone, and no author names or institutional affiliations are given either.

“Can a frozen VLM agent improve its spatial reasoning through parameter-update-free self-evolution, without depending on external expert spatial tools at inference time?”

— the authors