VibePrevisBench tests 11 coding agents on building 4D Blender scenes

VibePrevisBench tests 11 coding agents on building 4D Blender scenes

A paper introduces VibePrevisBench, a benchmark for 4D scene reconstruction with coding agents. The setup: an agent receives a brief and turns it into an executable 4D scene, then renders a proxy video of it. The scene is built in Blender.

The background is video generation. The authors note that controllable real-world video generation models can condition on a rendered graybox proxy video, which fixes layout, motion and camera. Coding agents can build such videos by reconstructing a 4D scene in Blender. But the authors say no existing benchmark checks both these videos and the long, multi-shot scenes behind them. VibePrevisBench is meant to fill that gap.

The benchmark has 242 tasks across four input kinds: scripts, photos, continuations and reference videos. Each delivery is scored along two evaluation dimensions. The first is Scene Check, which is rule-based: it re-runs the agent's code and compares the exported 4D scene with hidden ground truth facts in nine dimensions. The second is Screen Check, which is VLM-based: a VLM judge is asked whether the facts that passed Scene Check can be recognized in the proxy video.

The authors analyzed the evaluation results and trajectories of 11 frontier coding agents and report three observations. First, multimodal understanding separates the agents, especially in understanding space and camera motion. Second, all agents seem to already master basic camera work and staging, and fail at challenging requests. Third, agents inspect their own renders but fail to correct their errors.

From this the authors conclude that agents need better visual understanding of space and camera, explicit checks for challenging requests, and item-by-item verification of renders against the brief.

Key facts

  • VibePrevisBench has 242 tasks across four input kinds: scripts, photos, continuations and reference videos.
  • An agent turns a brief into an executable 4D scene in Blender and renders its proxy video.
  • Scoring has two parts: rule-based Scene Check against hidden ground truth facts in nine dimensions, and VLM-based Screen Check on whether passing facts are recognizable in the video.
  • The authors analyzed 11 frontier coding agents; multimodal understanding, especially of space and camera motion, separates them.
  • Agents inspect their own renders but fail to correct their errors.

Why it matters

Video generation models that take a rendered graybox proxy video as input get layout, motion and camera fixed by that proxy. Coding agents can build those proxies by reconstructing a 4D scene in Blender. The authors say no benchmark checks both the proxy videos and the long, multi-shot scenes behind them, and VibePrevisBench is their attempt to cover both.

Who it affects

Developers of coding agents and people who evaluate them are the direct audience, since the benchmark measures agents on a multimodal, visual task rather than on text-only coding. It also bears on anyone who wants to build proxy videos for controllable video generation by having an agent write Blender scenes from scripts, photos, continuations or reference videos.

How to use it

The practical guidance in the paper is aimed at agent builders. The authors say agents need better visual understanding of space and camera, explicit checks for challenging requests, and item-by-item verification of renders against the brief. The abstract gives no code, dataset release or license information, so how to run the benchmark is not stated.

How solid is it

The design is concrete: 242 tasks, four input kinds, a rule-based check that re-runs the code, and a VLM-judged check on the rendered video. Eleven frontier coding agents were analyzed. But the abstract names no authors or institutions, does not name the 11 agents, and gives no per-agent scores or rankings. It reports no numeric results, so the three observations cannot be checked against figures here. The second observation is itself hedged: all agents "seem to" already master basic camera work and staging.

Risks and caveats

The nine Scene Check dimensions are not listed, and the VLM judge is not named, so it is unclear how much Screen Check depends on the judge. The abstract does not say that Blender is the only supported tool. The findings come from one benchmark and one set of 11 agents, and the headline conclusions about what agents need are the authors' own reading of those results.

“Third, agents inspect their own renders but fail to correct their errors.”

— VibePrevisBench paper abstract