Test-Time Baking speeds up decoder-only view synthesis by up to 23x

Decoder-only Large View Synthesis Models (LVSMs) with a KV-cache have recently reached state-of-the-art quality. They produce novel views by regressing them directly from neural networks, without reconstructing 3D geometry. The authors point out a trade-off: Test-Time Training (TTT) layers save computation, but they sacrifice quality.
The paper proposes Test-Time Baking (TTB), a strategy that learns to bake global scene information from cross-view attention into a lightweight MLP. The aim is fast and high-fidelity decoding of novel views.
The authors describe two keys. The first is to repurpose the fast-weight MLP of TTT so that it learns the cross-view attention mapping explicitly, from the attention's input-output pairs, rather than replacing attention entirely. According to the authors, this gives the same rendering efficiency as TTT with substantially higher quality.
The second key deals with a modality misalignment between context prefilling and novel-view rendering. The authors split context tokens into two groups: camera-only context rendering tokens, which serve as the TTB input, and context ground-truth tokens that carry image content, which serve as the TTB mapping target. They say this relieves the baked MLPs of the modality generalization challenge between context and novel-view processing.
The headline result is up to 23x rendering speedup, with quality on par with the state of the art across several benchmarks. The claim is parity, not an improvement in quality. A project page is listed at test-time-baking.github.io.
Key facts
- Test-Time Baking (TTB) bakes global scene information from cross-view attention into a lightweight MLP, for fast decoding of novel views.
- The first key reuses the fast-weight MLP of Test-Time Training to learn the cross-view attention mapping from its input-output pairs, instead of replacing attention.
- The second key splits context tokens into camera-only rendering tokens (TTB input) and ground-truth tokens with image content (TTB target) to fix a modality misalignment.
- The authors report up to 23x rendering speedup with quality on par with the state of the art across several benchmarks.
- The paper positions TTB against TTT layers, which save computation but sacrifice quality.
Why it matters
Decoder-only LVSMs already reach state-of-the-art quality without reconstructing 3D geometry, and rendering speed is the part the authors set out to improve. Existing Test-Time Training layers cut computation but lose quality. TTB aims to keep the efficiency while recovering the quality, by keeping attention's behaviour and baking it into a small MLP instead of replacing attention.
Who it affects
Mainly researchers working on view synthesis and novel-view rendering, especially with decoder-only LVSMs and KV-cache decoding, and anyone studying Test-Time Training layers. The source is a research abstract and names no products or deployments.
How to use it
There is nothing to install from the abstract itself. The authors list a project page at https://test-time-baking.github.io. Practitioners would need to read the full paper to see how the baked MLPs are trained and applied.
How solid is it
This is a preprint abstract, so every figure is the authors' own claim. The 23x number is a maximum ("up to"), not a typical or average speedup, and the baseline it is measured against is not stated. The benchmark names, quality metrics and absolute rendering times are not given in the text.
Risks and caveats
The quality claim is parity with the state of the art, not a gain. The 23x figure is the best case, and without the baseline, hardware or benchmark details it is hard to judge what it means in practice. Treat it as a promising result to check against the full paper.
“We propose Test-Time Baking (TTB), a strategy that learns to bake global scene information from cross-view attention into a lightweight MLP, enabling fast and high-fidelity novel-view decoding.”
— Paper abstract, Hugging Face Papers