Glonce

VC-Attention speeds up low-bit video attention by up to 3.6x

researchThe Glonce editorsHugging Face🔥75Glonce rating75Impact75Novelty80Relevance95Surprise50Players25Credibility85
VC-Attention speeds up low-bit video attention by up to 3.6x

Video generation with Diffusion Transformers spends most of its compute on attention over long spatiotemporal sequences, so a low-bit attention kernel that is both accurate and fast is central to deploying these models cheaply. Xingyang Li and colleagues identify two separate obstacles to that goal and address both in a new framework called VC-Attention. On accuracy, a quantized block's scale is set by its largest values, so ordinary entries get squeezed into a narrow range of representable numbers. Prior methods smooth this out for queries and keys, but the authors find that outliers in the values do not follow any fixed channel or spatiotemporal pattern and remain the dominant source of error in the output. On speed, low-bit Tensor Cores only accelerate the two matrix multiplications inside attention, leaving the high-precision exponential computed in between as the longest stage of the pipeline on datacenter GPUs. VC-Attention pairs two training-free techniques to fix this. V-Smooth reorders value tokens with lightweight online clustering so that tokens sharing a hardware block quantize well together; it then quantizes only the residual left after subtracting the block's mean, and recovers that mean later from the row sum that online softmax already tracks, adding no extra bookkeeping. ExpCast-FP8 maps scores directly from log-domain to E4M3 FP8 probability codes with a single fused multiply-add, cutting out both the FP32 exponential and a separate format-conversion step. The authors implement VC-Attention on five GPUs (B200, B300, H200, RTX PRO 6000, RTX 5090) and test it on four video models (Wan2.2, LongCat-Video, HunyuanVideo-1.5, MiniMax-H3). Against BF16 FlashAttention-4, the attention kernel itself runs 1.46 to 1.59 times faster on datacenter Blackwell and Hopper cards and 2.3 to 3.6 times faster on workstation cards; generating a full clip end to end is 1.13 to 1.19 times and 1.36 to 1.70 times faster, though the source does not specify which of the two end-to-end ranges maps to which hardware class. The paper also reports that VC-Attention improves fidelity over other low-bit baselines, without giving a numeric accuracy figure for that claim.

Key facts

  • VC-Attention is a training-free low-bit attention framework for video Diffusion Transformers, combining value smoothing (V-Smooth) with a fused FP8 softmax cast (ExpCast-FP8).
  • V-Smooth reorders value tokens via online clustering and quantizes only the residual after subtracting the block mean, restoring that mean from the row sum online softmax already keeps.
  • ExpCast-FP8 maps log-domain scores directly to E4M3 FP8 codes in one fused multiply-add, removing the FP32 exponential and a separate format conversion.
  • Implemented on B200, B300, H200, RTX PRO 6000 and RTX 5090, and tested on Wan2.2, LongCat-Video, HunyuanVideo-1.5 and MiniMax-H3.
  • Versus BF16 FlashAttention-4, the attention kernel is 1.46-1.59x faster on datacenter Blackwell/Hopper GPUs and 2.3-3.6x faster on workstation cards, with end-to-end clip generation 1.13-1.19x and 1.36-1.70x faster.

Why it matters

Attention is the dominant cost of generating video with Diffusion Transformers because the sequences are long and span both space and time. Making a low-bit kernel work for that attention step needs both accuracy and speed, and the authors show these have separate causes: accuracy is limited by value outliers that follow no fixed pattern, and speed is limited by the high-precision exponential inside softmax, which low-bit Tensor Cores cannot accelerate.

Who it affects

Teams building or serving video generation models, and anyone optimizing GPU inference cost for Diffusion Transformers, whether on datacenter cards or workstation GPUs.

How to use it

VC-Attention is training-free, so it does not require retraining or fine-tuning the underlying model. The authors implement it for five GPUs, B200, B300, H200, RTX PRO 6000 and RTX 5090, and evaluate it on four video models, Wan2.2, LongCat-Video, HunyuanVideo-1.5 and MiniMax-H3. The source gives no pricing, license or availability details.

How solid is it

The framework is tested across five GPU types and four video models, with attention-kernel and end-to-end clip speedups reported for each hardware class against the same baseline, BF16 FlashAttention-4. The fidelity claim, that VC-Attention beats other low-bit baselines, is stated qualitatively without a numeric accuracy or error metric, and no comparison baseline other than BF16 FlashAttention-4 is given for the speed numbers.

Risks and caveats

The source does not say which of the two end-to-end clip-generation speedup ranges, 1.13-1.19x or 1.36-1.70x, applies to datacenter versus workstation hardware. It also does not specify the bit-width used for the low-bit attention scheme itself, and gives no author affiliation, institution or publication venue in the article text.