ATAS gives agentic video generation scene and event memory, tested on four generators
The paper starts from a problem in long-form agentic video generation: a screenplay has to become shots that stay spatially and temporally consistent even when the requests are not consecutive. Passing prompts or recent frames from one shot to the next, the authors say, gives no shared spatial frame for revisiting a scene and no time-indexed record for replaying an earlier event.
Their answer is Any Time, Any Space (ATAS). It first compiles a screenplay into shot requests, each carrying explicit location, camera and event-time references. Before a shot is generated, ATAS retrieves relevant evidence from two stores. The scene memory holds registered static geometry. The event memory holds dynamic observations indexed by scene, event and time. After generation, ATAS reconstructs and registers the clip. Memory is updated only when review and state checks accept the observation, and film delivery stays independent of memory admission, so a rejected clip does not block the film.
On results, the authors report that across four video generators ATAS improves all reported cross-shot consistency measures and achieves the highest aggregate benchmark score for each backbone. It also outperforms the compared agents and 3D-guided methods on target-image spatial revisit, leads three event-recapture criteria, and receives the highest human ratings on all six criteria. Video results and code are available on the project page at https://lhmd.top/atas/.
Key facts
- ATAS compiles a screenplay into shot requests with explicit location, camera and event-time references.
- Before generation it retrieves evidence from a scene memory (registered static geometry) and an event memory (dynamic observations indexed by scene, event and time).
- After generation it reconstructs and registers the clip, and memory is updated only when review and state checks accept the observation.
- Across four video generators, the authors report improvements on all reported cross-shot consistency measures and the highest aggregate benchmark score for each backbone.
- ATAS leads three event-recapture criteria and gets the highest human ratings on all six criteria; code and video results are on the project page.
Why it matters
Long videos made from many separate generation calls tend to lose track of where things are and what already happened. The paper's argument is that handing prompts or recent frames from shot to shot cannot fix this, because neither gives a shared spatial frame for returning to a scene or a time-indexed record for replaying an earlier event. ATAS treats the world state as the central object: a scene memory for geometry and an event memory for what happened and when, consulted before every shot.
Who it affects
Researchers and engineers building agentic pipelines that turn a screenplay into a multi-shot film. The framework is described as working across four video generators, so it is framed as a layer around a generator rather than a new generator itself.
How to use it
The authors say video results and code are available on the project page at https://lhmd.top/atas/. The abstract does not say whether the code is open-source under any specific license. In practice the method wraps an existing video generator: screenplay compilation before, memory retrieval before generation, reconstruction and registration after.
How solid is it
The claims come from the authors' own abstract and cover four video generators, a benchmark aggregate, a spatial revisit test, event-recapture criteria and human ratings on six criteria. The abstract gives no numeric scores, percentages or margins of improvement. The four video generators are not named, and neither are the benchmark and the compared agents and 3D-guided methods. The number or profile of human raters is not stated.
Risks and caveats
No limitations, compute cost or generation time are mentioned in the abstract, so the price of the extra reconstruction, registration and review steps is unknown. Because no margins are reported, the size of the gains cannot be judged from this text. The design also depends on review and state checks deciding what enters memory, and the abstract does not describe how those checks perform.
“Passing prompts or recent frames between shots does not provide a shared spatial frame for revisiting a scene or a time-indexed record for replaying an earlier event.”
— Abstract of the ATAS paper