StreamVibe: a benchmark and streaming model for video agents that infer implicit user intent
A paper listed on Hugging Face Papers, titled "StreamVibe: Proactive Video Agents for Evolving User Intent", starts from a gap in vision-language models. These models are moving from offline, passive understanding of video toward online, proactive interaction. The authors say prevailing methods mostly depend on explicit requests and miss implicit user intent. Their example is a user navigating a website who is effectively asking "where should I click next?" without saying so. The authors add that existing methods also fail to update their state as a user's requests keep changing, which makes them a poor fit for intent-aware interaction in practice.
The paper offers three pieces to address this. The first is StreamVibe-Bench, a new benchmark that serves as a testbed for user intent interaction. It has 4.1K evaluation samples and 85K training samples, covering explicit, implicit and dynamic intents.
The second is StreamVibe-VL, built on that benchmark. It is a streaming proactive model that infers implicit user intents from continuous observations and tailors its interactions as those intents evolve.
The third is StreamVibe-Harness. It keeps user intents as persistent states with flexible lifecycles, using a hierarchical memory to track the latest user intent and the state of the environment.
The authors report three experimental findings. (i) StreamVibe-VL keeps strong general video understanding while improving response timing and answer quality by 10.7% over other baselines. (ii) Existing proactive models remain limited in understanding user intent and in deciding when to respond, especially under implicit and dynamic intents. (iii) StreamVibe-Harness further improves response consistency and adaptation over sustained interaction, giving a 12.3% gain in proactive interaction. The text does not say whether the two percentages are percentage points or relative gains, or which metrics and baselines they refer to.
Key facts
- StreamVibe-Bench is a new benchmark for user intent interaction with 4.1K evaluation samples and 85K training samples, spanning explicit, implicit and dynamic intents.
- StreamVibe-VL is a streaming proactive model that infers implicit intents from continuous observations and adjusts its interactions as intents evolve.
- StreamVibe-Harness keeps user intents as persistent states in a hierarchical memory, tracking the latest user intent and environmental state.
- The authors report that StreamVibe-VL improves response timing and answer quality by 10.7% over other baselines while preserving general video understanding.
- The authors report that adding StreamVibe-Harness gives a 12.3% gain in proactive interaction.
Why it matters
Most video assistants answer what they are asked. This paper targets the harder case, where the user never states the need, as with someone browsing a website who wants to know where to click next. It also targets requests that shift over time. The authors say existing proactive models are still weak at understanding intent and at deciding when to respond, especially under implicit and dynamic intents. A benchmark with training and evaluation samples for these cases gives a way to measure that weakness.
Who it affects
The work is aimed at researchers building streaming and proactive vision-language models and video agents. The paper describes interaction settings such as a user navigating a website, but names no other use cases or products.
How to use it
The source mentions no code, dataset or model release, license or links, so there is nothing to try yet. For now it is a description of an approach: a benchmark for training and testing intent-aware models, a streaming model, and a hierarchical-memory harness that holds the current user intent and environment state across a long interaction.
How solid is it
These are the authors' own claims, taken from the paper's abstract-style summary on Hugging Face Papers. The 10.7% and 12.3% figures are given without baselines, metrics or a statement of whether they are percentage points or relative gains, so they cannot be checked from this text. No authors or institutions are named in the source, and no publication date is given.
Risks and caveats
The base model behind StreamVibe-VL is not named. The source also gives no information on latency, compute cost or real-world deployment, which matters for a model that must respond in a streaming setting. The benchmark was introduced alongside the model it favours, and the reported gains are self-reported.