Mosaic trains agents across multiple sessions, beating broadcasting on BrowseComp

Most agentic reinforcement learning methods assume single-session trajectories, where the agent interacts within one continuously accumulated context. The authors point out that techniques such as context compaction and subagent forking may produce trajectories that span several sessions. That creates a mismatch between the single-session assumption used in training and the way the agent actually runs at inference time.

To close the gap, the paper introduces Multi-Session Agentic Reinforcement Learning, a more general setting that supports an arbitrary number of sessions within a trajectory. In this setting, the authors say, a widely adopted approach is to broadcast the trajectory-level advantage to every session. Their criticism is that broadcasting does not distinguish the contributions of different sessions and does not guide how sessions are allocated within a trajectory.

The proposed fix is Mosaic, a multi-session agentic RL method with critic-free hierarchical optimization. It works at two levels. At the trajectory level, it folds session-usage signals into advantage estimation. At the session level, it shapes each session's advantage according to that session's estimated marginal contribution.

Experiments were run on BrowseComp with Qwen3.5-4B. According to the authors, Mosaic outperforms broadcasting in both the context compaction and the subagent forking scenarios, and it does so while using fewer sessions on average. Further analyses, they say, show improvements in session utility and allocation, which they suggest is a promising target for future advances in multi-session agentic RL. The summary gives no benchmark scores and does not say how large the reduction in session count is.

Key facts

The paper defines Multi-Session Agentic Reinforcement Learning, a setting that allows an arbitrary number of sessions within one trajectory.
The motivation: context compaction and subagent forking may produce multi-session trajectories, while most agentic RL training assumes a single session.
Mosaic is critic-free and optimizes at two levels: trajectory-level advantage estimation with session-usage signals, and session-level advantage shaping by estimated marginal contribution.
On BrowseComp with Qwen3.5-4B, Mosaic outperforms the broadcasting baseline in both context compaction and subagent forking scenarios while using fewer sessions on average.
No benchmark scores or percentage gains are given in the summary.

Why it matters

Agents that compact their context or fork subagents already work across several sessions at inference time, yet most agentic RL trains them as if everything happened in one accumulated context. The paper names this mismatch and proposes a training setting and method built for it. Its main target is credit assignment: broadcasting one trajectory-level advantage to all sessions treats every session as equally responsible for the outcome.

Who it affects

Mainly researchers and engineers who train long-horizon or web-browsing agents with reinforcement learning, especially those whose agents rely on context compaction or subagent forking. The experiments use a small model, Qwen3.5-4B, so the work is relevant to teams training compact agent models.

How to use it

The summary mentions no code release, so there is nothing to download from it. The practical takeaway is conceptual: if your agent runs across several sessions at inference, consider training with session-aware advantages rather than broadcasting a single trajectory-level signal, as Mosaic does with session-usage signals and per-session marginal contribution.

How solid is it

This is a paper abstract published on huggingface.co. The reported claim is narrow: Mosaic beats broadcasting on BrowseComp with Qwen3.5-4B in two scenarios, using fewer sessions on average. Only one benchmark and one model are named. No scores, percentage improvements or size of the session reduction are given, so the strength of the result cannot be judged from the summary alone.

Risks and caveats

Results on other models or benchmarks are not stated. The comparison named is against broadcasting only. The authors describe improved session utility and allocation as a promising target for future work, which frames it as an open direction rather than a settled result.