Temporal Context Routing aligns AI video and dialogue with script timing

Joint audio-video generation models, AI systems that generate video and audio together, have made real gains in visual quality and in keeping the two tracks synchronized with each other. What they still lack is precision: these models still offer only limited control over when shot transitions occur and when dialogue is spoken. That gap matters for script-driven content creation, where a structured script specifies exact timing for its beats, and timing errors can undermine narrative coherence and the viewing experience even when the audio and video match each other perfectly.

The paper traces this to how current joint generators handle time. Video and audio are aligned to each other on a shared temporal axis, but the script's own timing information, which shot happens when, which line is spoken when, exists only in the text of the prompt and is never mapped onto that shared axis. The result is that a model can produce audio and video that stay synchronized with each other while both drift away from the timeline the script specified.

To close that gap, the paper introduces Temporal Context Routing (TCR). TCR maps the script's specified timing onto the same shared temporal axis used for video and audio generation, then routes each part of the prompt's guidance to the matching position in both modalities. An instruction like 'this shot starts at second X' or 'this line is spoken at second Y' becomes a real temporal target for the model to hit, instead of ordinary prompt text it can drift away from.

Tested on 200 test scripts against a baseline the paper does not otherwise identify, TCR cuts Shot Boundary MAE, the mean absolute error in shot-transition timing, by 96%, from 1.11 seconds down to 0.042 seconds. Dialogue Acc@0.5s, the share of dialogue lines whose spoken timing lands within 0.5 seconds of the script, rises from 28.3% to 84.1%. The paper reports these gains without a drop in visual quality or in audio-visual synchronization compared with the baseline, and a separate user study found participants preferred TCR's output over the baseline's on all five evaluated dimensions, though it does not say what those five dimensions measure.

The abstract does not name the authors or an institution, does not identify the baseline model beyond calling it 'the baseline', does not say where the 200 test scripts came from, and states no publication venue, date, peer-review status, or release of code, weights, or a demo.

Key facts

TCR cuts Shot Boundary MAE (mean absolute error in shot-transition timing) by 96%, from 1.11 seconds to 0.042 seconds, on 200 test scripts.
Dialogue Acc@0.5s rises from 28.3% to 84.1% with TCR versus the baseline, on the same 200 test scripts.
TCR works by mapping a script's shot and dialogue timing onto the shared temporal axis that joint audio-video generators already use, rather than leaving that timing only in the prompt's text.
TCR holds visual quality and audio-visual synchronization comparable to the baseline while improving both timing metrics.
A user study found participants preferred TCR over the baseline on all five evaluated dimensions, though the paper does not describe what those dimensions measure.

Why it matters

Script-driven content creation depends on precise timing: a shot cut that lands late, or dialogue spoken off the beat a script called for, breaks the pacing the script was written to deliver, even when the audio and video are perfectly synced to each other. Current joint audio-video generators can produce well-synchronized clips but still offer only limited control over whether shot transitions and dialogue actually land where a script says they should, because the script's own timing lives only in the prompt text and is never mapped onto the timeline the model builds. TCR targets exactly that gap: aligning generation with the script's own clock, not only with itself.

Who it affects

Anyone turning a structured, timed script directly into finished audio-video, for example in an automated production pipeline, needs this kind of alignment to keep the output faithful to the script's intended pacing rather than merely internally consistent. The paper does not name a company, product, or institution behind TCR, and the abstract does not name its authors either.

How to use it

The abstract gives no publication venue, date, or peer-review status, and it does not mention released code, model weights, or a public demo. There is currently no stated way to run, license, or reproduce TCR beyond the numbers reported in the paper.

How solid is it

The comparison is quantitative and holds across two different timing metrics: on 200 test scripts, TCR cuts Shot Boundary MAE by 96% and lifts Dialogue Acc@0.5s from 28.3% to 84.1%, while visual quality and audio-visual synchronization reportedly stayed comparable to the baseline. A user study adds a human check on top of those metrics, with participants preferring TCR over the baseline on all five evaluated dimensions. Two gaps limit how far the evidence generalizes: the paper never identifies what the baseline model actually is beyond the label 'the baseline', and it does not say whether the 200 test scripts are newly built or drawn from an existing benchmark.

Risks and caveats

Beyond the unnamed baseline and untraced test scripts, the abstract does not describe what the user study's five evaluated dimensions actually measure, and it states no peer-review status, publication venue, or release of code, weights, or a demo. Until those are available, the result rests on the paper's own reported numbers rather than on independent reproduction.

“video and audio may remain synchronized with each other while both fail to follow the script timeline”

— the paper