Velox speeds up video world model attention 4.36 times at chunk 48
Video world models turn actions and camera motion into long visual rollouts, which makes video a natural interface for interactive simulation. That use needs responsive generation, but a coherent rollout also has to remember what it has already generated. Chunked video diffusion models usually keep this memory in growing KV caches, so each later chunk is slower than the last. The paper targets that bottleneck.
The authors introduce Velox, a training-free attention router. Instead of attending to the whole history, it selects context at region level, using frustum slab correspondences from streaming geometry. Velox represents historical blocks as camera frustum slabs with depth intervals, and it obtains the depth of the current block through a recency-priority multiwarp. For regions that past blocks cover well, the overlap between frustum slabs softly modulates each layer's model-derived block importance. Regions that are not covered keep the model-derived route, with no geometric bias added.
The design is built to fit existing hardware kernels. The fixed token-block granularity matches FlashAttention-style tiled execution and reduces the attention term that grows with rollout length. Frustum slab construction runs asynchronously with denoising on the same H200.
The reported results: at chunk 48, Velox accelerates attention by 4.36 times and denoising by 2.43 times. On the full 158-case WBench navigation split, it stays within 0.15 API-free average points of the dense LingBot-Fast-Cam baseline. The abstract points readers to a project page at velox-world.com/velox for more details.
Key facts
- Velox is a training-free attention router for chunked video diffusion world models; it picks context at region level using frustum slab correspondences from streaming geometry.
- At chunk 48, it accelerates attention by 4.36 times and denoising by 2.43 times.
- On the full 158-case WBench navigation split, it stays within 0.15 API-free average points of dense LingBot-Fast-Cam.
- Well-covered regions get their model-derived block importance softly adjusted by frustum slab overlap; uncovered regions keep the model-derived route unchanged.
- Fixed token blocks match FlashAttention-style tiled execution, and slab construction runs asynchronously with denoising on the same H200.
Why it matters
Interactive video world models need to respond quickly, yet they must also keep a visual memory of what they have generated. The usual way to keep that memory is a growing KV cache, which makes later chunks progressively slower. Velox attacks the attention cost that grows with rollout length by letting each region draw only on the history that geometrically overlaps it, rather than on everything. It does this without any training.
Who it affects
The work speaks to researchers and engineers building chunked video diffusion world models for interactive simulation, where camera motion and actions drive long rollouts. The comparison is made against the dense LingBot-Fast-Cam model, so teams working with that kind of dense chunked setup are the most direct audience.
How to use it
The abstract describes Velox as a training-free router, so it is presented as an addition to an existing chunked video diffusion model rather than something that needs retraining. It works with fixed token blocks that match FlashAttention-style tiled execution, and its frustum slab construction runs asynchronously with denoising. The abstract points to a project page at https://velox-world.com/velox/ for more details.
How solid is it
The evidence is the abstract's own reported numbers: 4.36 times faster attention and 2.43 times faster denoising at chunk 48, and a result within 0.15 API-free average points of dense LingBot-Fast-Cam on the full 158-case WBench navigation split. These are the authors' claims, taken from a short abstract; the underlying experiments are not described there.
Risks and caveats
Speedups are reported only at chunk 48, with no figures for other chunk indices. The abstract gives no absolute baseline scores on WBench and does not say whether the 0.15-point gap is a loss or a gain, only that Velox stays within it. No memory-usage figures are given, and the only hardware mentioned is a single H200 with the number of GPUs not stated.
“Chunked video diffusion models usually keep this context in growing KV caches, so later chunks become progressively slower.”
— Velox paper abstract