Glonce

LEASH routes computer-use agents to a strong model only when a weak one is unsure

agentsThe Glonce editorsHugging Face🔥60Glonce rating60Impact45Novelty60Relevance100Surprise30Players15Credibility75
LEASH routes computer-use agents to a strong model only when a weak one is unsure

Computer-use agents increasingly take on long-horizon tasks. Running a frontier model at every step is expensive, while a small open-weight model on its own often fails. Weak-to-strong escalation is the middle path: a weak agent acts by default and hands control to a strong one only when needed. The hard part is deciding when to hand over, which calls for an uncertainty signal cheap enough to check at every step. A second problem remains after the handoff: the strong agent can still get lost, and the black-box API it is usually served through exposes no internals to monitor.

The paper proposes LEASH, a training-free framework that, in the authors' words, unleashes the strong agent only when the weak agent's latent uncertainty signals it is needed, and keeps it on a leash afterward. It has three layers: (1) initial task uncertainty for early routing, (2) step-level uncertainty for runtime escalation, and, after handoff, (3) a shadow observer for continued oversight of the strong black-box agent. Latent uncertainty drives all three.

The uncertainty signal works by comparing the weak model's hidden states with those from its past successful and failed trajectories. A state that looks more like the failures signals higher uncertainty. While the weak agent runs, the estimator reuses activations already produced during action generation, so it needs no extra forward pass and no dedicated monitor training.

The experiments are on OSWorld, across four contemporary model backbones. LEASH consistently improves over standalone weak agents and outperforms random escalation by 6.55 to 11.94 points at matched escalation rates. With Qwen3.5-35B-A3B, LEASH reaches 73.66% task success at $0.4117 per task. That matches always-on Claude Opus 5, which scores 72.90%, while reducing inference cost by 19.2%.

Key facts

  • LEASH is a training-free framework: a weak open-weight computer-use agent acts by default and escalates to a strong black-box agent only when its latent uncertainty signals the need.
  • It has three layers: initial task uncertainty for early routing, step-level uncertainty for runtime escalation, and a shadow observer that oversees the strong agent after handoff.
  • Uncertainty comes from comparing the weak model's hidden states with those of its past successful and failed trajectories, reusing activations already produced during action generation, with no extra forward pass.
  • On OSWorld across four model backbones, LEASH beats random escalation by 6.55 to 11.94 points at matched escalation rates.
  • With Qwen3.5-35B-A3B it reports 73.66% task success at $0.4117 per task, against 72.90% for always-on Claude Opus 5, with inference cost reduced by 19.2%.

Why it matters

Long-horizon computer-use tasks force a trade-off: a frontier model at every step is costly, and a small open-weight model alone often fails. LEASH targets that gap by spending the strong model only where the weak one looks unsure. The reported result is a weak-plus-escalation setup that matches an always-on Claude Opus 5 baseline on OSWorld (73.66% against 72.90%) at 19.2% lower inference cost. It also addresses what happens after handoff, where a black-box API gives no internals to inspect.

Who it affects

Teams that run computer-use agents on long tasks and pay for frontier-model calls at each step are the obvious audience. So are those who use a small open-weight model as the default agent and want a principled rule for when to call a stronger black-box model.

How to use it

The source describes the method but does not mention a code or model release. The idea itself is simple to state: let the weak agent run, track how closely its hidden states resemble those from its past failed trajectories rather than successful ones, route early if the task looks risky, escalate at a step when uncertainty climbs, and keep a shadow observer on the strong agent after handoff. The estimator needs no dedicated monitor training and no extra forward pass.

How solid is it

The numbers come from the authors' own experiments on OSWorld across four model backbones. The gain over random escalation (6.55 to 11.94 points) is measured at matched escalation rates, which makes it a like-for-like comparison. The headline cost comparison is for one backbone, Qwen3.5-35B-A3B. The source does not name the other three backbones or give their results, and it does not state the escalation rate behind the 73.66% figure. The per-task cost of always-on Claude Opus 5 is not given, so the 19.2% reduction cannot be checked from the text alone.

Risks and caveats

No evaluation outside OSWorld is mentioned. The source describes no limitations or failure cases. The gap between 73.66% and 72.90% is under one point, so the claim is parity with the always-on baseline rather than a clear win on accuracy; the advantage reported is cost. The shadow observer is meant to address the strong agent getting lost after handoff, but the source gives no separate results for that layer.

“a training-free framework that unleashes the strong agent only when the weak agent's latent uncertainty signals it is needed, and keeps it on a leash afterward”

— LEASH paper abstract