InfinityEdit enables infinite editing of streaming video

Instruction-based video editing, built on large pretrained models, has improved a great deal, but nearly all existing methods share the same assumption: the edited output is aligned with a fixed source clip, frame by frame, over a fixed time span. That works when the input is a video you already have in full. It breaks down for open-ended streams, such as restyling a live game feed as it plays or applying a camera move to a shot that is still being generated. In those cases an edit has to extend into frames that do not exist yet, rather than being applied once to a static clip.

The paper names this problem infinite video editing and defines it precisely: given a preceding segment of the stream and a new edit request, a model must generate the next segment so that it continues the stream while carrying out the edit. The process repeats as an unbounded sequence of edit instructions arrives over time. The authors identify two challenges this creates: each edit has to be a faithful continuation of the stream rather than a frame-wise rewrite, and generation quality has to stay stable even as edits keep accumulating.

To address it, the authors first built a data-collection pipeline specifically for infinite video editing, then used the resulting data to train InfinityEdit, a lightweight adapter that adds unbounded editing ability to a streaming video generator (the text does not name which one). The adapter has three attention modules: history cross-attention, which guides the frames being denoised using the input frames; temporal causal self-attention, which keeps temporal information flowing only from earlier frames to later ones; and edit cross-attention, which injects the edit request into generation. At inference time the adapter is switched on only for the chunk in which an edit request arrives; every chunk after that is generated by the original streaming model, starting from a reset anchor frame. That design applies the requested edit without giving up the base model's ability to keep generating indefinitely. The authors report that, in extensive experiments, InfinityEdit faithfully continues the stream under each edit and stays stable across unbounded sequences of edits, though the text does not give specific benchmark figures or comparisons to back that claim.

Key facts

The paper defines a new task, infinite video editing: generating the next segment of an ongoing video stream so that it continues the stream while applying a new edit request, repeated as an unbounded sequence of edit instructions arrives.
It contrasts this with existing instruction-based editing methods, which assume in-place editing, aligning the edited video with a fixed source clip frame by frame over a fixed time span, a pattern that fails for open-ended streams such as restyling a live game feed or adjusting the camera move in an ongoing shot.
The proposed method, InfinityEdit, is a lightweight adapter with three attention modules: history cross-attention, temporal causal self-attention, and edit cross-attention.
At inference, the adapter activates only for the chunk in which an edit request arrives; every following chunk is generated by the original streaming model from a reset anchor frame, so the edit is applied without losing the base model's ability to keep generating indefinitely.
The authors report that in extensive experiments InfinityEdit faithfully continues the stream under each edit and stays stable over unbounded edit sequences, though the text gives no benchmark figures or baseline comparisons to support that claim.

Why it matters

Nearly all current instruction-based video editing assumes the full clip already exists: the system aligns the edited output with the source, frame by frame, across a fixed span. That assumption fails as soon as the video is still being generated or streamed, for example a live game feed being restyled, or a shot whose camera move is being adjusted while it plays. Infinite video editing reframes the task for exactly that case: continue the stream while applying the edit, then keep doing so as further edit requests keep arriving. It is as much a definitional contribution as a technical one, giving a name and a concrete formulation to a gap that instruction-based editing had not addressed.

Who it affects

The work targets teams building streaming or live AI video generation: anyone who wants to apply an edit instruction mid-stream instead of regenerating a finished clip. Because InfinityEdit is built as an adapter added to an existing streaming video generator rather than a new generator of its own, it is aimed at groups that already run such a system and want editability without retraining or replacing it. It has no direct bearing on editing pre-recorded footage, which is a different, already well-served problem.

How to use it

InfinityEdit activates only when an edit request arrives. Its three attention modules split the job: history cross-attention keeps the newly generated frames grounded in what came before, temporal causal self-attention lets information flow forward in time only, and edit cross-attention feeds the edit instruction into generation. Once the edited chunk is produced, the adapter switches off and the original streaming model resumes from a reset anchor frame, so the stream keeps running indefinitely rather than needing a fresh restart after every edit. This is a research paper, not a shipped product: the text gives no license, pricing, or availability details, and does not mention releasing code, model weights, or the training dataset.

How solid is it

The evidence in the text is a claim, not a shown result: the authors state that extensive experiments found InfinityEdit faithfully continues the stream under each edit and stays stable over unbounded edit sequences, but the text does not include benchmark scores, metrics, or comparisons against baseline methods to back that up. The paper also does not name the base streaming video generator it was tested on, or give the scale of the data-collection pipeline behind it. It appears on Hugging Face's papers listing with 30 points and no comments yet, credited to Yunze Tong, and the text itself carries no affiliation or venue, so there is little external signal to weigh alongside the authors' own claim.

Risks and caveats

Beyond the missing benchmark numbers, the scope is narrow: infinite video editing is a specific problem within video generation, not a general editing tool. Nothing in the text says which base model the adapter was built for, which limits how directly the result generalizes to other streaming generators. No code, weights, or dataset has been mentioned as released, so the method cannot currently be verified or reproduced independently. And because the paper is credited to Yunze Tong but names no institution, there is no track record to weigh the claims against beyond the paper itself.