Programmable World Model separates world state from video generation

Recent video world models can generate increasingly realistic, interactive visual experiences, but the authors argue they lack reliable ways to hold onto persistent world state or enforce programmable rules across long interactions. To address this, they introduce Programmable World Model, a framework that decouples world-state evolution from visual observation generation instead of asking a single video model to track everything implicitly.

In this framework, an agent translates natural-language instructions into executable programs that specify entity states and state-transition rules, giving users direct control over individual entities and how they interact. A lightweight engine then runs these programs to update and maintain an explicit, persistent global world state, one that includes off-screen entities and non-visual attributes, not just what is currently on camera.

To connect that explicit state to the rendered video, the authors introduce state-augmented 3D oriented bounding boxes (OBBs) as an intermediate representation. Combined with the target camera trajectory, the OBBs are deterministically compiled into pixel-aligned spatiotemporal conditioning signals that drive a pretrained video model, which the text does not name, acting as the generative renderer. The stated payoff is that users can build playable games with predefined mechanics, direct control over individual entities, and persistent world state that holds throughout gameplay.

The authors also introduce CombatStateBench, a benchmark for evaluating programmable world models; the text gives no detail on how it is built, such as scenario count or task types, beyond its name. On CombatStateBench, Programmable World Model reaches 94% Count Accuracy and 98% State Accuracy, which the authors describe as substantially outperforming existing interactive video world models while supporting coherent long-horizon generation, though no baseline figures or margin are given for the systems it is compared against. They conclude that separating explicit state evolution from generative rendering is effective for building persistent, programmable worlds. The abstract names no authors or institutions, and it states no release date, code or dataset availability, or publication venue.

Key facts

Programmable World Model decouples explicit, rule-based world-state evolution from visual generation, instead of relying on a single video model to track state implicitly.
An agent turns natural-language instructions into executable programs that define entity states and state-transition rules, and a lightweight engine runs those programs to maintain a persistent global world state, including off-screen entities and non-visual attributes.
State-augmented 3D oriented bounding boxes, combined with the target camera trajectory, are deterministically compiled into pixel-aligned spatiotemporal conditioning signals for a pretrained video model, not named in the text, that renders the scene.
On the newly introduced CombatStateBench benchmark, the method reaches 94% Count Accuracy and 98% State Accuracy, which the authors describe as substantially outperforming existing interactive video world models while supporting coherent long-horizon generation.
The design is meant to let users build playable games with predefined mechanics, direct control over individual entities, and world state that persists throughout gameplay.

Why it matters

Video-based world models have gotten good at rendering realistic, interactive scenes, but according to the authors they still cannot reliably hold onto persistent state or enforce programmable rules once an interaction runs long: entities that leave the frame have no guaranteed way to keep their state, and there is no deterministic way to script game logic on top of what the video model generates. Programmable World Model addresses this by pulling world state out of the video model entirely. An agent turns natural-language instructions into executable programs that define entity states and transition rules, and a separate, lightweight engine runs those programs to keep an explicit, persistent global world state, including entities that are off-screen and attributes that are never directly visible. That state is then compiled, through state-augmented 3D oriented bounding boxes and the target camera trajectory, into conditioning signals that a pretrained video model uses to render the scene. Splitting state from rendering this way is the paper's central claim to novelty.

Who it affects

The work targets researchers and developers building interactive, AI-generated video environments: anyone trying to create a playable game or simulation on top of a generative video model rather than a conventional game engine, and anyone who needs entities and rules to persist reliably across a long interactive session. It also gives researchers evaluating this class of system a concrete new benchmark, CombatStateBench, to measure against. The text describes no consumer product, release, or general-audience use; this is a research contribution, not a shipped tool.

How to use it

There is no pricing or licence to speak of: this is a research paper, not a product. Mechanically, the described pipeline runs from natural-language instructions, through an agent that compiles them into executable programs, to a lightweight engine that runs those programs to track world state; state-augmented 3D oriented bounding boxes plus a target camera trajectory are then compiled into conditioning signals for a pretrained video model that renders the final frames. The text does not name that underlying video model, and it states no release date, no information on code or dataset availability, and no publication venue, so there is no way to run or obtain this system from what is written here.

How solid is it

The only evidence offered is a single benchmark, CombatStateBench, which this same paper introduces, so there is no independent evaluation to check it against. On that benchmark, Programmable World Model reaches 94% Count Accuracy and 98% State Accuracy, and the authors say it substantially outperforms existing interactive video world models while supporting coherent long-horizon generation. They give no baseline numbers or margin for the systems being compared, though, and no detail on how CombatStateBench itself is built, such as scenario count, task types, or data source, beyond its name. That makes the headline numbers hard to check independently.

Risks and caveats

The abstract names no authors or institutions, so there is no way from this text alone to assess who did the work or where. The pretrained video model used for rendering is not identified. No release date, code or dataset availability, or publication venue is given, so reproducing or using this work is not currently possible from the source alone. And because both the accuracy numbers and the outperformance claim rest on a benchmark the authors built themselves, with no outside baseline figures supplied, the reported gains should be read as self-reported until independently confirmed.

“These results demonstrate the effectiveness of separating explicit state evolution from generative rendering for building persistent, programmable worlds.”

— the authors of the Programmable World Model paper