AnyGameRenderer turns rough game footage into polished video

AI coding agents can now build small playable games in minutes, the authors write, but the results usually look like programmer art: gray blockouts, colored geometric primitives and flat semantic blobs. The paper presents AnyGameRenderer, a video-to-video generative renderer that takes such footage and produces high-quality visuals with characters, motion detail and effects. Throughout, it keeps the layout, camera and every entity's position and timing intact.

The input is plain video. The system needs no engine G-buffers, per-object class labels or low-poly proxies, and gray blockouts, colored primitives, semantic maps and wireframes all work as input. The authors say it generalizes across genres that agents build: tower defense, auto-battlers, top-down survivors, turn-based battles, kart racing and crowd runners. It also preserves entity correspondence: group counts stay correct, and entities appear and disappear on the right frame.

The renderer is built by LoRA fine-tuning MiniMax-H3 ref2va, a 33B reference-video-conditioned video generation model. Training uses "sim10k" paired clips, a figure the source gives in that form without explaining the token. The clips come from three sources: multi-stream rendering data from 38 agent-made games exported from GEAR, an AI game-creation tool; two-stream rendering data from Unreal Engine, with programmatically controlled crowds, count changes and entity scales; and real gameplay recordings, with pairs reconstructed via 3D-box detection.

To measure entity correspondence, the authors propose an entity-level benchmark of 59 held-out cases with 753 entities, scored against ground-truth boxes. Compared with the base model, AnyGameRenderer raises temporal match from 50.9% to 62.9% and cuts missed count changes from 34.7% to 21.3%. The authors also say fewer entities are hallucinated or left unrendered, but give no figures for that.

Key facts

AnyGameRenderer is a video-to-video renderer that turns gray blockouts, colored primitives, semantic maps and wireframes into polished visuals while keeping layout, camera and every entity's position and timing.
It is made by LoRA fine-tuning MiniMax-H3 ref2va, a 33B reference-video-conditioned video model, on paired clips from 38 agent-made GEAR games, Unreal Engine data and real gameplay recordings.
It needs only plain video as input: no engine G-buffers, per-object class labels or low-poly proxies.
The new entity-level benchmark has 59 held-out cases with 753 entities, scored against ground-truth boxes.
Against the base model, temporal match rises from 50.9% to 62.9% and missed count changes fall from 34.7% to 21.3%.

Why it matters

Coding agents can already produce playable prototypes quickly, but the visuals stay crude. This work targets that gap: it restyles the rough footage without moving anything, so the game's state (where each entity is and when it appears) survives the makeover. The paper also tackles a specific weakness of generative video, which is losing count of things. Its benchmark scores entities directly rather than judging overall look.

Who it affects

The paper frames the problem around games built by AI coding agents and the people who use them. The listed genres are tower defense, auto-battlers, top-down survivors, turn-based battles, kart racing and crowd runners. Researchers working on controllable video generation get an entity-level evaluation setup built from 59 cases and 753 entities.

How to use it

The abstract gives no usage instructions. The stated input is plain video of a game, whether gray blockouts, colored primitives, semantic maps or wireframes, with no engine-side data needed. The source mentions no release of code, weights or the benchmark.

How solid is it

This is a paper abstract, and the figures are the authors' own. The benchmark is also the authors' proposal. The comparison is only against the base MiniMax-H3 ref2va: temporal match 50.9% to 62.9%, missed count changes 34.7% to 21.3%. No comparison with any model other than the base model is given, and no human evaluation or visual-quality metric is reported.

Risks and caveats

Gains are shown on a 59-case benchmark that the authors designed themselves, and even after tuning, temporal match is 62.9%. No figures are given for hallucinated or unrendered entities. No inference speed, real-time capability or compute cost is stated, so it is unclear whether this suits interactive use. No breakdown of how many clips come from each of the three data sources is given, and the abstract does not explain what "sim10k" means beyond "sim10k paired clips".

“AI coding agents can now build small playable games in minutes, but the results usually look like programmer art”

— AnyGameRenderer paper abstract