TurnWise brings turn-level credit to streaming video RL

Streaming video understanding asks a model to make a decision again and again as video arrives: respond now, or keep observing. A recent line of work trains such models with Group Relative Policy Optimization (GRPO). According to the paper's authors, those approaches put their effort into reward design while keeping a single trajectory-level advantage shared across all turns.

The authors argue this shared credit is a problem. It can reinforce redundant replies after earlier successes, and it can discourage useful follow-ups after earlier failures. In other words, one outcome for the whole trajectory is applied to every turn, even though each turn is a separate decision.

The paper proposes TurnWise, a turn-level GRPO framework meant to align learning with individual streaming decisions. The authors say it addresses the mismatch between trajectory-level feedback and turn-level interaction, promoting informative responses and reducing redundancy.

TurnWise has two parts. The first is turn-level credit assignment. It uses what the authors call silence-completed prefix scoring to construct group-relative advantages from each decision's remaining return, which separates that decision's learning signal from earlier rewards. The second is turn-mean policy optimization. It averages token losses within each turn and then across turns, so response decisions and waiting decisions carry equal weight regardless of how long the output is.

The authors state that these complementary designs improve streaming policy learning without a learned critic or additional rollouts. They also state that extensive experiments show the method outperforms prior RL approaches for streaming video understanding. The abstract does not give benchmark names, scores or the size of the improvement.

Key facts

TurnWise is a turn-level GRPO framework for reinforcement learning on streaming video understanding, where a model repeatedly decides whether to respond or keep observing.
It targets a flaw the authors see in existing GRPO approaches: a single trajectory-level advantage shared across turns, which can reinforce redundant replies and discourage useful follow-ups.
Turn-level credit assignment uses silence-completed prefix scoring to build group-relative advantages from each decision's remaining return.
Turn-mean policy optimization averages token losses within each turn, then across turns, so response and waiting decisions get equal weight regardless of output length.
The authors say the method needs no learned critic and no additional rollouts, and that it outperforms prior RL approaches; no figures are given.

Why it matters

Streaming video models do not answer one question about a finished clip. They have to keep deciding when to speak and when to stay silent. The authors' point is that training signals built for whole trajectories fit that setting poorly, because an early success or failure bleeds into later decisions. TurnWise tries to give each respond-or-wait decision its own credit, which is a change to the training objective rather than to reward design.

Who it affects

Mainly researchers training streaming or online video understanding models with GRPO-style reinforcement learning. The abstract names no authors, institutions, models or products, so it does not say who has adopted the approach.

How to use it

The abstract describes two components that would be added to a GRPO setup: turn-level credit assignment with silence-completed prefix scoring, and turn-mean policy optimization. The authors say they work without a learned critic or additional rollouts. No code, weights or release is mentioned in the source.

How solid is it

The evidence here is the authors' own statement that extensive experiments show TurnWise outperforms prior RL approaches for streaming video understanding. The source gives no benchmark names, scores, size of improvement, or list of the prior approaches compared against, so the claim cannot be checked from the text. This is a summary of the abstract, not independent verification.

Risks and caveats

The claim of better performance is unquantified in the source. The harm from shared credit is described as something that can happen, not as a measured rate. No model names or sizes and no training data are mentioned, so how well the method carries across setups is unknown from the abstract alone.

“Such shared credit can reinforce redundant replies after earlier successes and discourage useful follow-ups after earlier failures.”

— TurnWise paper abstract