Invoker-Render pairs two video models for controllable streaming generation

Invoker-Render pairs two video models for controllable streaming generation

A paper listed on Hugging Face Papers under the title "Code World Model 2.0: Towards Infinite Controllable Video Generation" presents a renderer called Invoker-Render. The abstract starts from the idea that visual proxies can separate explicit world-state simulation from photorealistic video synthesis. That gives a controllable interface for both offline clip creation and real-time interactive streaming.

The authors say a practical renderer must solve two problems. First, the upstream proxies vary, from coarse layouts to fine articulated geometry, depending on the interaction. Second, streaming inference has to work in a few causal steps without accumulating drift across the model's own self-generated history.

Invoker-Render pairs a bidirectional creation model with a causal four-step streaming model. Both sit under a shared conditioning interface over structural proxies, an appearance reference and language. The system was trained on an aligned, approximately 200-hour multi-granularity corpus. According to the abstract, it honors fine geometric constraints when they are supplied and synthesizes plausible local motion when only coarse proxies are given, so the level of input detail can be chosen at runtime.

To connect bidirectional creation with real-time streaming without compounding step compression and rollout drift, the authors introduce Progressive Forcing (ProForcing). It works in two stages. It first trains a causal predictor and distills it into a four-step renderer, using histories derived from training videos. Training then shifts to continuous self-generated rollouts. A pipelined serving stack supports efficient streaming inference.

The abstract says experiments show versatile control across proxy families, granularities and camera paths, consistent gains over recent streaming baselines, and stable 30-minute racing rollouts that preserve scene and subject appearance.

Key facts

  • Invoker-Render pairs a bidirectional creation model with a causal four-step streaming model under a shared conditioning interface over structural proxies, an appearance reference and language.
  • It was trained on an aligned, approximately 200-hour multi-granularity corpus, and it handles inputs from coarse layouts to fine geometry, with the level of input detail selectable at runtime.
  • Progressive Forcing (ProForcing) first trains a causal predictor and distills it into a four-step renderer using histories from training videos, then moves to continuous self-generated rollouts.
  • The authors report stable 30-minute racing rollouts that preserve scene and subject appearance, and consistent gains over recent streaming baselines.

Why it matters

The paper targets a split between two uses of generated video: offline clip creation and real-time interactive streaming. Its approach is to let visual proxies carry the world-state simulation while the renderer handles photorealistic synthesis. The hard part it names is streaming in only a few causal steps without drift building up in self-generated history, and ProForcing is its answer to that.

Who it affects

The work is aimed at people building controllable video generation and interactive streaming renderers, where input from an upstream simulation or layout has to be turned into photorealistic video. The abstract frames the proxies as inputs from coarse layouts to fine articulated geometry.

How to use it

The abstract describes a conditioning interface that takes structural proxies, an appearance reference and language, with the amount of input detail chosen at runtime. No release of code, weights or demo is mentioned.

How solid is it

This is a preprint abstract, and the claims are the authors' own. The reported results are qualitative in form: versatile control across proxy families, granularities and camera paths, consistent gains over recent streaming baselines, and stable 30-minute racing rollouts. No quantitative benchmark results are given, and the gains are not specified by metric, size or baseline name.

Risks and caveats

No frame rate, resolution, latency or hardware figures are given for streaming. The abstract does not say the model is real-time at any specific speed; it names real-time interactive streaming as a use case. The racing rollouts are not described further, with no game or dataset named. The abstract does not mention the name "Code World Model" or "2.0" from the title; it names the system Invoker-Render.

“Visual proxies decouple explicit world-state simulation from photorealistic video synthesis, offering a controllable interface for both offline clip creation and real-time interactive streaming.”

— Paper abstract