LynnReal-Omni unifies video generation for agentic visual workflows

Researchers have presented LynnReal-Omni, a native multimodal video generation framework built on a 32B shared multimodal diffusion transformer. The stated problem is that video diffusion models are stochastic and hard to control: getting precise content often requires repeated sampling with no guaranteed success, and long scenes drift in appearance, interactions and temporal coherence over time. Agentic visual creation tools, such as explicit references, editable 3D scenes or executable game states, can give more stable control but do not by themselves guarantee that objects or characters stay faithful to a reference. The authors argue that combining the two approaches can produce stable, high-quality generation, and built LynnReal-Omni to do that in a single model rather than as separate tools.

The 32B model unifies seven capabilities that are normally handled by separate systems: text-to-video generation, image-conditioned generation, reference-guided generation, structural control, editing, restoration of degraded video, and long-video generation. It accepts heterogeneous inputs, including appearance references, editable 3D renders and game recordings, so that an agent can combine several kinds of visual conditioning within one model instead of chaining separate tools.

Alongside the base model, the team trained a dedicated 27B parameter variant called LynnReal-Omni-Flash, built for real-time rendering. Flash further cuts inference cost through model and decoding acceleration, including a lightweight VAE decoder. On a single H100 GPU, warm generation and decoding of a 22-frame, 540p video takes 843 milliseconds with the base LynnReal-Omni model and 377 milliseconds with Flash, according to the paper, more than halving latency.

To train and check the system, the authors built a data pipeline covering video cleaning, subject association, multimodal annotation and aligned control construction, producing a curated corpus of multi-shot audiovisual segments. They also introduce MSAVP, an evaluation design of 100 prompts and 20 metrics that separately scores instruction following, plausibility of what is generated, visual quality, temporal behavior and audio coordination, rather than folding all of it into a single score. The authors describe the resulting system as a foundation for real-time streaming video generation and a unified, controllable, efficient basis for agentic visual creation. The paper does not name any competing model for comparison, does not state the size of the training data or the training compute used, and does not mention whether code, model weights or a public demo will be released.

Key facts

LynnReal-Omni is a 32B-parameter shared multimodal diffusion transformer that unifies text-to-video, image-conditioned generation, reference-guided generation, structural control, editing, degraded-video restoration and long-video generation in one model.
A separate 27B-parameter 'Flash' variant is trained specifically for real-time rendering, using model and decoding acceleration including a lightweight VAE decoder.
On one H100 GPU, warm generation and decoding of a 22-frame, 540p video takes 843 ms with the base model versus 377 ms with Flash.
The authors built a dedicated data pipeline (video cleaning, subject association, multimodal annotation, aligned control construction) and a new evaluation suite, MSAVP, with 100 prompts and 20 metrics covering instruction following, plausibility, visual quality, temporal behavior and audio coordination.
The paper does not disclose training data size or compute cost, does not compare against named competing models, and does not state whether code, weights or a demo will be released.

Why it matters

Most video generation systems force a choice between free-form diffusion, which is flexible but hard to control precisely, and agentic tools like 3D scenes or game engines, which give control but do not guarantee that generated content stays faithful to what was specified. LynnReal-Omni's pitch is to fold both into one model so an agent can mix references, 3D renders and structural controls as inputs to a single system rather than stitching together separate specialized tools for each task.

Who it affects

The work targets people building agentic visual pipelines, meaning systems where an AI agent composes video from multiple kinds of input rather than a single text prompt: game and simulation content generation, video editing tools driven by 3D or reference assets, and applications that need to restore or extend degraded footage as part of an automated workflow.

How to use it

The paper describes the architecture, training pipeline and evaluation methodology (MSAVP) but does not state whether model weights, code or a public demo are being released, so at this stage the work is documented research rather than an available product.

How solid is it

The paper reports concrete latency numbers measured on a single H100 GPU (843 ms for the base model and 377 ms for Flash, on a 22-frame 540p video) and introduces a 100-prompt, 20-metric evaluation design meant to separate different aspects of video quality rather than collapsing them into one score. However, the abstract does not include comparative results against named competing systems, nor does it state the training data size or compute budget used, so the claims of unification and efficiency cannot be checked against a baseline from this text alone.

Risks and caveats

The source text supplies no author names, institutional affiliations, publication date, training data details, or release plans for code or weights; none of that should be assumed. The reported speed and quality figures come from the authors' own paper and have not been independently verified or compared against rival systems in the text available.