Glonce

LongHorizon-Harness raises Qwen 3.7-Plus's WeaveBench score to 80.7%

researchThe Glonce editorsHugging Face🔥59Glonce rating59Impact68Novelty62Relevance95Surprise48Players30Credibility85
LongHorizon-Harness raises Qwen 3.7-Plus's WeaveBench score to 80.7%

Researchers propose LongHorizon-Harness, a framework built around a different way of running long-horizon LLM agents: tasks that need sustained reasoning, tool use, and revision across many interdependent steps. Their diagnosis of existing agent harnesses is specific: task execution, task state, and completion assessment all live inside one growing context window, which makes the state hard to track and lets a wrong self-assessment made early carry forward into later decisions uncorrected.

LongHorizon-Harness instead keeps task state explicit and outside the execution context, updating it only with facts independently verified from the environment rather than the agent's own account of what it did. The mechanism is a Manage-Execute-Audit (MEA) loop with three roles: a manager that maintains the task state and decides the next subtask, a fresh-context executor that carries out that subtask without the accumulated clutter of prior steps, and a read-only auditor that checks the resulting environment state before the loop moves to the next round. A lightweight component called AgentAdapter lets the harness swap in different underlying models and agent harnesses as interchangeable backends, without modifying how those backends natively run their own agent loops.

The paper reports gains on three benchmarks. Applied to Qwen 3.7-Plus, LongHorizon-Harness raises WeaveBench performance from 51.8% to 80.7%, Terminal-Bench 2.1 from 69.7% to 77.2%, and OSWorld 2.0 from 2.8% to 8.3%. Applied to Claude Opus 4.7 on a subset of OSWorld 2.0, it raises the score from 20.0% to 34.3%. The authors present this as evidence of consistent gains across different models, different underlying harnesses, and different interaction domains, from coding-style terminal tasks to open-ended computer-use environments.

Key facts

  • LongHorizon-Harness separates task-state tracking from task execution and verifies state changes against the environment, instead of trusting an agent's in-context self-assessment.
  • Its Manage-Execute-Audit (MEA) loop splits the work into a manager (state and subtask selection), a fresh-context executor (carries out the subtask), and a read-only auditor (verifies the result).
  • On Qwen 3.7-Plus, it lifts WeaveBench from 51.8% to 80.7%, Terminal-Bench 2.1 from 69.7% to 77.2%, and OSWorld 2.0 from 2.8% to 8.3%.
  • On a subset of OSWorld 2.0, it raises Claude Opus 4.7's score from 20.0% to 34.3%.
  • AgentAdapter lets the harness plug in different models and agent harnesses as backends without changing their native agent loops.

Why it matters

The paper targets a specific failure mode in long-horizon agents: when task execution, task state, and completion assessment all sit inside one growing context, the state becomes hard to track and a wrong self-assessment made early can propagate uncorrected into later decisions. LongHorizon-Harness reframes long-horizon execution as a task-state management problem and answers it by keeping state explicit, external to execution, and updated only from facts independently verified against the environment rather than the agent's own narration.

Who it affects

The framework targets builders and researchers of agents that run many interdependent steps over time, such as terminal and coding agents and open-ended computer-use agents, the kind of workloads represented by Terminal-Bench and OSWorld in the paper's own evaluation.

How to use it

AgentAdapter is designed as a lightweight interchange layer: it lets a different model or a different underlying agent harness be swapped in as the backend without modifying that backend's native agent loop. The paper demonstrates this by plugging in both Qwen 3.7-Plus and Claude Opus 4.7 as backends under the same MEA loop.

How solid is it

The reported gains are consistent in direction across all four measurements: three benchmarks with Qwen 3.7-Plus and one with Claude Opus 4.7, spanning WeaveBench, a terminal-task benchmark, and open-ended computer-use environments (OSWorld). The source text does not name the authors, their institution, or a publication venue, does not compare LongHorizon-Harness by name against other existing long-horizon harnesses, and gives no figures for training or inference cost.

Risks and caveats

The benchmark numbers come from the authors' own paper with no independent replication described in the source. The added manager and auditor roles mean more model calls per task step than a single-agent loop, and the source gives no data on that overhead or on the added latency and cost of running MEA instead of a simpler harness. No comparison to specific competing long-horizon frameworks is given, so it is not possible to say from this text how the gains stack up against other proposed fixes for the same problem.