FastFrames paper: 1.3B video model runs at 36 ms per frame

FastFrames paper: 1.3B video model runs at 36 ms per frame

A paper listed on Hugging Face introduces FastFrames, a family of fast autoregressive video diffusion models. These models generate frames in response to new observations and inputs, which the authors call a promising medium for interactive visual experiences. Their complaint is speed: existing models, they write, are orders of magnitude too slow for fluid interaction.

The authors profiled the inference pipeline of state-of-the-art models. They found that operations often treated as fixed implementation details, such as VAE decoding and KV cache updates, in fact account for over half of end-to-end inference latency. The paper challenges the assumption that these components are indispensable and replaces them with lightweight alternatives.

Two variants are reported. The 1.3B variant runs at 36 ms per frame, or 28 responses per second (RPS), on a single B200 GPU. The authors say it matches the generation quality of a state-of-the-art model that is nearly 8x slower. The 14B variant runs at 45 ms per frame, or 22 RPS, on an 8x B200 GPU node, and the authors say it yields significantly better visual quality and realism.

The paper says FastFrames enables real-time interaction in applications such as camera-conditioned and physics-conditioned video generation. Its headline claim is that the models reach interactive latency without compromising quality.

The source is the paper's abstract. It does not name the slower state-of-the-art model, does not say what the lightweight replacements are, and does not state what the 14B quality gain is measured against.

Key facts

  • Profiling of state-of-the-art autoregressive video diffusion models found that VAE decoding and KV cache updates together account for over half of end-to-end inference latency.
  • FastFrames replaces both components with lightweight alternatives rather than treating them as fixed.
  • The 1.3B variant runs at 36 ms per frame (28 RPS) on a single B200 GPU and, per the authors, matches the quality of a state-of-the-art model that is nearly 8x slower.
  • The 14B variant runs at 45 ms per frame (22 RPS) on an 8x B200 GPU node and, per the authors, yields significantly better visual quality and realism.
  • The authors demonstrate real-time interaction in camera-conditioned and physics-conditioned video generation.

Why it matters

Autoregressive video diffusion models produce frames as new inputs arrive, which makes them candidates for interactive experiences. The authors argue that existing models are orders of magnitude too slow for that. FastFrames targets the bottleneck: the paper's profiling puts VAE decoding and KV cache updates at over half of end-to-end latency, and the authors replace them instead of accepting them as given. At 36 ms per frame for the 1.3B variant and 45 ms per frame for the 14B variant, the reported numbers are in the range the authors call interactive.

Who it affects

The paper points to people building interactive video generation, where a model reacts to user input frame by frame. The examples it names are camera-conditioned and physics-conditioned video generation. The reported figures rely on B200 GPUs: one for the 1.3B variant and an 8x B200 node for the 14B variant.

How to use it

The abstract describes applications rather than a product: camera-conditioned and physics-conditioned video generation with real-time interaction. No release of code, weights, or a demo is mentioned. For now this is a result to read and track, not something to install.

How solid is it

The evidence is the authors' own abstract on a Hugging Face papers page, so every claim is theirs. The latency figures are specific: 36 ms per frame (28 RPS) on one B200 and 45 ms per frame (22 RPS) on an 8x B200 node. The quality claims are looser. The 1.3B model is said to match a state-of-the-art model that is nearly 8x slower, but that model is not named. The 14B model's quality gain has no stated baseline, and no quality metrics or scores are given.

Risks and caveats

No video resolution, frame size, or context length is given for the latency figures, so the numbers are hard to compare with other systems. The abstract does not say what the lightweight replacements for VAE decoding and KV cache updates are. It also gives no individual latency shares for the two components, only that together they exceed half. The 14B result needs an 8x B200 GPU node, not a single GPU. The abstract names no authors or institutions.

“existing models are orders of magnitude too slow for fluid interaction”

— FastFrames paper abstract