World in World lets frozen video models explore new camera views

Autoregressive video world models can already support interactive, long-horizon exploration of a recorded scene, but controlling that exploration flexibly has remained hard. Sending the generated rollout to a new viewpoint requires several things at once: staying synchronized with the recorded event, placing content that was actually observed into the newly requested view, plausibly filling in regions of the scene that the new viewpoint exposes for the first time, and, when the exploration later revisits an earlier position, reproducing the appearance the model itself generated there instead of inventing something different. Existing methods typically meet these requirements with task-specific modules or by giving the underlying model additional training.

The authors present World in World, a training-free, inference-time interface that adds these capabilities to an existing autoregressive video model without any additional training or fine-tuning; the model's weights stay completely frozen throughout. The interface works by converting several kinds of control evidence into clean, camera- and time-labeled visual states that the frozen model reads through its own native self-attention, the same mechanism it already uses for any visual token, rather than through a separate bolted-on module.

Four sources of evidence feed the interface. Source-video observations supply what was actually recorded. Target-view scene projections show what the requested camera angle should contain, based on known geometry. Geometry renderings guide the model in plausibly completing regions of the scene that a camera move exposes for the first time. Generated states retrieved from beyond the model's rolling cache let a revisited location match what the model generated for it earlier, instead of being redrawn inconsistently. Each source carries what the authors call token-level support, meaning it applies only to specific tokens, and each has its own availability schedule rather than being present at every generation step.

Two mechanisms govern how this evidence gets used. A correspondence router combines persistent point identities with scene geometry to work out which generated tokens correspond to which source-video tokens, then steers queries that have a matching evidence source toward those tokens. Evidence-wise attention CFG (EWA) separately controls how much extra influence each auxiliary evidence channel contributes, using the attention responses produced during that same denoising forward pass rather than a separate step.

On one frozen backbone, the shared interface supports three distinct capabilities: camera-controlled rerendering, long-horizon revisiting, and human-motion transfer. The evaluation in the paper, however, covers only the first of these. World in World is tested on camera-controlled video rerendering under a range of viewpoint changes, judged on perceptual quality, temporal consistency, and camera-following accuracy. The text gives no numeric scores or percentages for any of the three criteria, and names no specific competing method, referring only to existing methods in general. It also does not name the training or evaluation dataset used, and it gives no author names, institutions, publication date, or information about released code, model weights, or a license.

Key facts

World in World is a training-free, inference-time interface: it adds new exploration capabilities to an autoregressive video model without fine-tuning, and the model's weights stay completely frozen throughout.
The interface converts four kinds of control evidence, source-video observations, projected target-view scenes, geometry renderings for newly exposed regions, and generated states retrieved from beyond the rolling cache, into camera- and time-labeled visual tokens read through the frozen model's own self-attention.
A correspondence router matches generated tokens to source-video tokens using persistent point identities plus scene geometry, while a separate mechanism, Evidence-wise attention CFG (EWA), independently tunes each evidence channel's contribution using attention from the same denoising pass.
The same frozen backbone and interface support three distinct capabilities: camera-controlled rerendering, long-horizon revisiting, and human-motion transfer.
The evaluation covers only camera-controlled video rerendering under varied viewpoint changes, judged on perceptual quality, temporal consistency, and camera-following accuracy; the text gives no numeric scores, no named competing methods, and no training or evaluation dataset.

Why it matters

Autoregressive video world models are already capable of open-ended, long-horizon exploration, but giving them precise, flexible control, whether a specific requested camera path or a consistent revisit of an earlier point, has usually meant building a task-specific module or retraining the model for that one capability. World in World's contribution is to show that several different control signals, including camera projections and geometry renderings, can instead be reformatted as ordinary visual tokens and handed to a model's existing, frozen self-attention. That turns adding a control capability into an inference-time engineering problem rather than a training problem, and it lets one frozen backbone pick up three separate capabilities, camera-controlled rerendering, long-horizon revisiting, and human-motion transfer, through the same interface.

Who it affects

The people this concerns most directly are researchers and engineers working with autoregressive, causal video world models, who gain a single training-free interface for camera control, scene revisiting, and motion transfer instead of building or training a separate module for each. The text names no company, lab, or product built on top of it, and it gives no author names or institutions at all, so who specifically produced the work cannot be said from the source.

How to use it

There is no product, price, or access route to describe: the text mentions no released code, model weights, dataset, or license, consistent with this being a research method rather than a shipped tool. What can be described is the mechanism itself. Control evidence, comprising source-video frames, projected target views, geometry renderings for regions a new camera angle exposes, and generated states retrieved from beyond the rolling cache, is turned into camera- and time-labeled visual tokens; a correspondence router lines these up with the right position in the video using tracked point identities plus geometry, and Evidence-wise attention CFG (EWA) separately dials each evidence channel's influence up or down using attention from that same denoising step. None of this touches the frozen model's weights.

How solid is it

The abstract states what was measured, camera-controlled video rerendering under a range of viewpoint changes, scored on perceptual quality, temporal consistency, and camera-following accuracy, but it reports no actual numbers, scores, or percentages for any of the three. It also names no specific competing method it was measured against, referring only to existing methods in general, and it does not name the training or evaluation dataset used. The other two stated capabilities, long-horizon revisiting and human-motion transfer, are not covered by this evaluation at all.

Risks and caveats

As written, this is a description of a method and an evaluation setup rather than a checkable result. No author names, institutions, or publication date appear in the text; there is no released code, model, or dataset to inspect; and there is no quantitative outcome to weigh the perceptual-quality and consistency claims against. Two of the three capabilities the interface is said to support, long-horizon revisiting and human-motion transfer, are asserted but not covered by the described evaluation, which is scoped only to camera-controlled rerendering.