A new survey maps how memory works in autoregressive video generation

A paper offers a systematic review of memory mechanisms in autoregressive (AR) video generation, the approach in which visual sequences are extended step by step through causal rollouts to enable long-horizon video, interactive world models, and evolving visual environments. The authors identify a core bottleneck: as a generated sequence grows longer, practical AR models still have to work within strictly bounded context windows, storage, and compute budgets. As a result, historical information that matters, such as entity identities, dynamic states, and causal changes triggered by interventions, frequently drops out of the active context well before it stops being relevant to what the model generates next. The paper treats keeping this information available over time as a fundamental memory problem for AR video generation. The authors formulate memory operationally as persistent historical information that is maintained across the outer autoregressive steps and can still influence future generation even after the original evidence that produced it is no longer locally accessible in the context. Building on this unified framing, the paper organizes existing work in the literature through five complementary perspectives: Forms, the representational carriers used to store history; Functions, the specific semantic and physical information that needs to be preserved; Operations, the full lifecycle of writing, reading, updating, managing, and integrating memory; Learning, how memory behaviors are optimized under closed-loop rollouts; and Evaluation, the paradigms used to diagnose whether a system has genuine memory capabilities rather than superficial consistency. The paper closes by synthesizing open challenges for the field, including designing memory architectures that are composable and resource-aware, making state updates trustworthy, enabling learning from self-generated rollouts, and establishing standardized evaluation. The authors describe the goal as bridging representations, mechanisms, and learning paradigms to give the field a structured foundation for building reliable, memory-conditioned video generation systems. The paper does not name specific AR video models it evaluates, does not report experiments or performance numbers, and does not identify its authors or institutions; it is a review of the literature rather than a study introducing a new model or benchmark.

Key facts

The paper is a systematic review of memory mechanisms in autoregressive (AR) video generation, not a new model or benchmark.
It defines memory operationally as persistent historical information maintained across outer AR steps that can still shape future generation after the original evidence has left the active context.
The core problem framed by the authors: practical AR models must run under strictly bounded context windows, storage, and compute, so information like entity identities, dynamic states, and intervention-induced causal changes can leave context before it stops mattering.
The literature is organized into five perspectives: Forms, Functions, Operations, Learning, and Evaluation.
Open challenges highlighted include composable and resource-aware memory architectures, trustworthy state updating, self-rollout learning, and standardized evaluation.

Why it matters

Autoregressive video generation extends visual sequences step by step, which is what makes long-horizon video and interactive world models possible, but the same step-by-step process forces every practical system to work under tight context, storage, and compute limits. The paper argues that this is precisely why models tend to lose track of things that still matter, such as who or what is in the scene, the state of a dynamic environment, or a causal change triggered earlier by an intervention. By defining memory as historical information that persists across steps and can influence generation even after its source evidence is gone from the active context, the paper gives the field a shared way to talk about a problem that previously showed up as ad hoc fixes in individual systems.

Who it affects

The review is aimed at researchers and engineers building or studying autoregressive video generation systems, including long-horizon video generators and interactive world models, who need a shared vocabulary and map of existing approaches to memory rather than a scattered set of individual techniques.

How to use it

The paper is a literature review rather than a tool or model, so its practical use is as a reference framework: readers can use the five-perspective organization (Forms, Functions, Operations, Learning, Evaluation) to categorize existing memory techniques, identify gaps in a specific system's approach, or decide where a new method fits. It does not provide code, weights, or a specific benchmark to run.

How solid is it

The account rests on the paper's own description of its scope and framework rather than on reported experiments; the source text is a review that surveys and organizes existing literature, and it does not itself present quantitative results, named models it benchmarks, or empirical comparisons. No authors, institutions, publication venue, or date are given in the source.

Risks and caveats

Because this is a survey rather than an empirical study, it offers a conceptual framework and a synthesis of open challenges rather than validated solutions or measured improvements. The authors themselves flag that composable and resource-aware memory architectures, trustworthy state updating, self-rollout learning, and standardized evaluation remain unresolved, meaning the field still lacks agreed-upon ways to verify whether a system's memory is working correctly rather than just appearing consistent.

“We formulate memory operationally as persistent historical information maintained across outer AR steps, capable of influencing future generation even after the originating evidence is no longer locally accessible.”

— the authors