Glonce

LiveAnimate streams real-time human animation at 19.63 FPS

researchThe Glonce editorsHugging Face🔥78Glonce rating78Impact72Novelty82Relevance100Surprise60Players45Credibility78
LiveAnimate streams real-time human animation at 19.63 FPS

Researchers presented LiveAnimate, a pose-driven human animation system that generates video of a target person from a single reference image and a stream of driving poses. The authors describe it as, to their knowledge, the first animation system to combine real-time streaming with stable long-form generation at billion-parameter scale. It is built on a 14B-parameter video Diffusion Transformer (DiT).

The authors note that existing diffusion-based animation systems take minutes to hours to render a single clip, which rules out responsive interaction for uses such as live streaming, telepresence and virtual avatars. LiveAnimate addresses this with a two-stage training pipeline: Reference-Anchored Teacher-Forcing Adaptation first converts a pretrained bidirectional DiT into a block-causal autoregressive generator, and Block-wise Self-Forcing Distillation then cuts the sampling budget to three steps. To keep a subject's appearance stable over long streams, the system adds Pose-Retrieval Sink Attention (PR-Sink), a bounded key-value cache made of a Static Sink that permanently anchors the first generated block, a Dynamic Sink that holds one pose-retrieved historical block, and a three-slot Rolling Window. When a pose recurs later in the stream, PR-Sink restores the matching appearance context instead of keeping the full history in memory, so both memory use and per-block latency stay constant no matter how long the stream runs.

Combined with Ulysses sequence parallelism and operator fusion, these components let LiveAnimate run streaming inference at 19.63 frames per second on two NVIDIA H100 GPUs. On a three-minute benchmark, the authors report that perceptual quality and subject identity stay nearly constant from the first 30 seconds through the final minute, while prior systems, according to the authors, either degrade substantially over the same rollout or need hours of offline computation to avoid it.

Key facts

  • LiveAnimate is a 14B-parameter video Diffusion Transformer built for pose-driven human animation from a single reference image.
  • A two-stage pipeline, Reference-Anchored Teacher-Forcing Adaptation followed by Block-wise Self-Forcing Distillation, converts a bidirectional DiT into a block-causal generator and cuts the sampling budget to three steps.
  • Pose-Retrieval Sink Attention (PR-Sink) keeps memory use and per-block latency constant over arbitrarily long streams via a Static Sink, a Dynamic Sink and a three-slot Rolling Window.
  • The system streams at 19.63 FPS on two NVIDIA H100 GPUs.
  • On a three-minute benchmark, quality and identity stay nearly constant from 30 seconds to the final minute, while the authors say prior systems degrade or need hours of offline rendering to avoid it.

Why it matters

Diffusion-based video generation has delivered strong quality but at a cost of minutes to hours per clip, which the authors say rules out responsive interaction. LiveAnimate is presented as the first system to combine real-time streaming with stable output at billion-parameter scale, aiming to close the gap between the visual quality large diffusion models produce and the latency interactive avatar applications require.

Who it affects

The paper frames the work around interactive applications: live streaming, telepresence and virtual avatars, so builders of avatar and streaming-animation products are the direct audience. Researchers working on autoregressive video diffusion, distillation and KV-cache design for long-form video generation are also a direct audience for the training pipeline and PR-Sink mechanism described.

How to use it

The source text does not state an institution, funding source, or whether code or model weights will be released, and it gives no licensing or pricing information. Based on what is available, it is not yet clear how, or whether, LiveAnimate becomes accessible outside the paper itself.

How solid is it

The reported results are concrete on throughput: 19.63 FPS streaming inference on two NVIDIA H100 GPUs, a three-step sampling budget after distillation, and a three-minute benchmark over which quality and identity are described as nearly constant from 30 seconds to the final minute. That quality claim, however, is qualitative; the source gives no numeric quality or identity score, and it does not name the prior systems used for comparison.

Risks and caveats

The source text names no institution or funding behind the work and gives no independent replication path, since code and model release are not mentioned. Because the stability claim over long streams rests on a qualitative description rather than a published metric, and the comparison systems are unnamed, the improvement over prior work is not independently verifiable from this text alone.