LayerRecall fixes long-horizon consistency in AI video generation

Autoregressive video diffusion models generate long videos chunk by chunk, drawing on a bounded window of recent context. That recency-based caching keeps nearby frames consistent, but it evicts older cues, so when a subject, object, scene or attribute reappears later in a video, the model has already forgotten what it looked like. Prior memory mechanisms gave models access to that nonlocal history, but the researchers found that access alone does not guarantee the history gets used well. Their analysis of video DiT (diffusion transformer) layers showed that different layers have distinct preferences for current, recent and distant context, meaning a long-range memory system has to decide not just what to retrieve, but which layers should use it.

The paper proposes LayerRecall, a state-conditioned, layer-selective memory router. It retrieves relevant historical key/value (K/V) states and injects them only into the specific backbone layers that are memory-sensitive, leaving local attention untouched elsewhere in the network. To train the router without depending on scarce high-quality long-horizon video data or hand-labeled memory-allocation targets, the authors also introduce Cross-Horizon Prediction Matching (CHPM), a supervision method that uses a privileged long-context reference to guide the bounded-memory router in prediction space.

Across 100 multi-shot evaluation prompts, LayerRecall posts the best overall results on the MemoBench and MovieBench benchmarks while matching its backbone model's performance on VBench-Long, which the authors read as stronger long-range recall without sacrificing local continuity. Qualitative analysis surfaces a memory-guided self-correction effect: when a local attribute is initially generated wrong, it snaps back to match its earlier appearance in the video without resetting the ongoing motion or scene structure. Further analysis reportedly shows the router transfers across different backbone architectures and adds only negligible inference overhead. The source text is the paper's abstract; it does not give numeric score deltas for the three benchmarks, name the authors or their institutions, or specify a release date or code availability.

Key facts

LayerRecall is a state-conditioned, layer-selective memory router for autoregressive long-video diffusion generation.
It retrieves relevant historical key/value states and injects them only into memory-sensitive backbone layers, leaving local attention elsewhere untouched.
A companion method, Cross-Horizon Prediction Matching (CHPM), supervises the router using a privileged long-context reference, avoiding the need for scarce long-horizon video data or manual memory labels.
Across 100 multi-shot evaluation prompts, it achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long.
The authors report memory-guided self-correction of mismatched attributes, cross-backbone portability and negligible inference overhead, though no numeric figures back these claims in the available text.

Why it matters

Long AI-generated videos tend to forget what things looked like once they scroll out of the model's recent context window: a character's shirt or a room's layout can silently change if it reappears minutes later. LayerRecall targets that specific failure mode by working out which network layers actually benefit from long-range memory and feeding them historical context selectively, rather than dumping all history everywhere. That selectivity is the core contribution: broad access to history already existed in prior memory mechanisms, but the paper's analysis found access alone did not translate into effective use.

Who it affects

The work is aimed at researchers and engineers building or fine-tuning autoregressive video diffusion systems, particularly anyone working on multi-shot or long-duration video generation where subjects, objects or scenes need to persist and reappear consistently over time. It also matters to teams building the benchmarks used to evaluate such consistency, since LayerRecall was tested against MemoBench, MovieBench and VBench-Long specifically.

How to use it

The source text is limited to the paper's abstract, so it gives no information on code release, model weights, licensing or availability. Architecturally, the method is described as portable across different backbone models, and the authors report negligible added inference cost, but neither claim comes with a supporting number in the available text.

How solid is it

The abstract states LayerRecall achieves the best overall results on MemoBench and MovieBench and matches its backbone on VBench-Long, evaluated across 100 multi-shot prompts, but gives no numeric score deltas for any of the three benchmarks, only the qualitative claim of leading results. The self-correction effect is described as observed through qualitative analysis rather than a quantified metric. No author names, institutional affiliations or publication venue appear in the available text.

Risks and caveats

Every figure in this account traces back to the paper's abstract alone; the source text supplies no benchmark scores, no quantified inference-overhead figure, no release date and no code or dataset availability. The claims of cross-backbone portability and negligible overhead are the authors' own framing, not independently verified numbers, and should be read as such until fuller results are published.

“LayerRecall achieves the best overall results on MemoBench and MovieBench while matching its backbone on VBench-Long, demonstrating stronger long-range recovery without sacrificing local continuity.”

— the paper's abstract