VideoCoCo writes Blender code as chain-of-thought for video generation
Text-to-video models produce visually impressive clips but often get physics wrong, because a generative model has to infer how a scene evolves over time from a short, compressed text prompt. Earlier chain-of-thought approaches tried to fix this by generating an intermediate plan or a set of visual states before the final video, but the researchers say these representations are typically non-executable or temporally sparse, so they cannot fully instantiate or control how a scene unfolds from start to finish.
VideoCoCo, an agentic dual-engine framework, replaces that sparse intermediate plan with executable code. Given a text prompt, a coding agent synthesizes a Blender program that explicitly specifies the scene and its temporal evolution. A simulation engine then runs that program to produce a deterministic spatiotemporal draft, essentially a rough but physically grounded animation. A separate generative video engine takes this draft and turns it into a photorealistic video through what the paper calls draft-conditioned editing. The split separates process-level reasoning about physics from the step that produces realistic visuals.
To adapt the video-editing engine to work on these simulated drafts rather than natural footage, the team built VideoCoCo-3K, a curated dataset of draft-instruction-target triplets. Tested against the OmniWeaving baseline, VideoCoCo raised the PhyGenBench score from 0.475 to 0.558 and the VBench-2.0 score from 52.18 to 77.88, and the authors report it reaches the best average score on both benchmarks. They conclude that executable code is an effective, controllable and inspectable intermediate representation for generating physically consistent video.
Key facts
- VideoCoCo has a coding agent write executable Blender code as a process-level chain of thought, replacing the non-executable or sparse plans used by earlier methods.
- A simulation engine runs that code into a deterministic spatiotemporal draft, which a generative video engine then edits into a photorealistic video (draft-conditioned editing).
- The team built VideoCoCo-3K, a curated dataset of draft-instruction-target triplets, to adapt the video editor to simulated drafts.
- Applied to the OmniWeaving baseline, VideoCoCo raised the PhyGenBench score from 0.475 to 0.558 and the VBench-2.0 score from 52.18 to 77.88.
- The authors report VideoCoCo achieves the best average score on both PhyGenBench and VBench-2.0.
Why it matters
Text-to-video models look convincing but frequently get physics wrong, since a network has to infer how a scene evolves purely from a short text prompt. Prior chain-of-thought fixes generated intermediate plans or keyframes, but those representations are usually not executable and cover only scattered moments, so they cannot pin down a scene's full timeline. VideoCoCo's novelty is swapping that sparse plan for executable Blender code: a program a simulation engine can actually run to produce a physically grounded draft before any pixel of the final video is generated.
Who it affects
This is a research contribution aimed at people building or evaluating text-to-video systems: model developers looking for ways to inject physical consistency, and researchers working with benchmarks like PhyGenBench and VBench-2.0. The paper does not describe a public product, so it does not directly affect end users of video-generation tools yet.
How to use it
The text does not state whether VideoCoCo's code, model weights or the VideoCoCo-3K dataset are being released, nor any pricing or licensing terms, so there is nothing to run today. It describes the architecture, a coding agent, a Blender simulation engine and a generative video engine, rather than a usage guide.
How solid is it
The core evidence is a benchmark comparison: applying VideoCoCo to the OmniWeaving baseline raised the PhyGenBench score from 0.475 to 0.558 and the VBench-2.0 score from 52.18 to 77.88, and the authors report the best average score on both benchmarks among the methods compared. The paper does not disclose compute cost, model size or runtime, and it does not explain the units or scale of the two benchmarks, which limits how far the numbers can be judged in absolute terms.
Risks and caveats
The source text names no authors or institutions and gives no publication or release date, so provenance cannot be checked from the text alone. It also leaves the scales of PhyGenBench (0 to 1) and VBench-2.0 (tens) unexplained, which limits how the improvement should be read. The whole approach depends on a coding agent correctly translating a text prompt into working Blender code, a step whose reliability and failure modes the text does not address.
“These results demonstrate that executable code provides an effective, controllable, and inspectable intermediate representation for physically consistent video generation.”
— the authors, in the paper