Glonce

DocAtlas beats human experts on MMLongBench-Doc benchmark

researchThe Glonce editorsarXiv🔥74Glonce rating74Impact68Novelty72Relevance100Surprise48Players45Credibility85

Long-document understanding, answering questions that require finding and combining evidence scattered across many pages, layouts, tables, figures, and charts, is typically handled one of two ways: retrieval-augmented systems that select evidence from a static index before generating an answer, or agentic systems that add multi-turn tool use but usually run on frozen proprietary backbones whose behavior is fixed by the prompt. The authors present DocAtlas, which instead treats long-document understanding as a mutable-state, information-seeking process. DocAtlas is built as a mutable document harness: an external environment that decides, at each step, what document information is searched, read, stored, reviewed, and shown to the model. Given a document and a question, the harness gives the model search, reading, note-taking, and review tools, and it maintains a hierarchical tree and a note store that both get updated as the agent records evidence, all within a fixed context budget.

The same harness works two ways: at inference time with large vision-language models, and as a training environment for end-to-end reinforcement learning of compact VLM agents. Using GPT-5.4 inside the DocAtlas harness reaches 71.4% accuracy on the MMLongBench-Doc benchmark, exceeding the benchmark's human-expert reference score of 65.8%. Separately, a Qwen3.5-4B VLM trained end-to-end with reinforcement learning inside the DocAtlas environment reaches 63.7% on the same benchmark, compared with a 54.4% baseline for the same model given the document directly without the harness. The authors present this gap as evidence that mutable document-harness design can improve compact document agents by a large margin.

Key facts

  • DocAtlas is an external 'mutable document harness' that controls, at each step, what document content is searched, read, stored, reviewed, and shown to the model, via search, reading, note-taking, and review tools over a hierarchical tree and note store.
  • With GPT-5.4, DocAtlas reaches 71.4% accuracy on the MMLongBench-Doc benchmark, above the benchmark's human-expert reference score of 65.8%.
  • A Qwen3.5-4B VLM trained end-to-end with reinforcement learning inside the DocAtlas environment reaches 63.7% on the same benchmark, versus a 54.4% baseline for the same model fed the document directly without the harness.
  • The authors contrast DocAtlas with two existing approaches: retrieval-augmented systems that pick evidence from a static index before generation, and agentic systems that add multi-turn tool use but typically run on frozen proprietary backbones whose behavior is set only by the prompt.
  • The same harness design supports both inference-time use with large VLMs and end-to-end RL training of compact VLM agents.

Why it matters

Long documents scatter the evidence a question needs across pages, layouts, tables, figures, and charts, and the two existing approaches each give something up: static retrieval fixes the evidence set before generation, while agentic tool use typically runs on a frozen backbone whose behavior is set only by the prompt, not by any persistent memory of what has already been found. DocAtlas instead builds a harness with state that changes as the agent works: a hierarchical tree and note store updated with each search, read, and note the model takes, under a fixed context budget. That design lets the model choose what to keep looking at rather than reasoning over a document snapshot fixed at the start.

Who it affects

The paper targets people building document-understanding agents: those who want a large VLM to read long documents more accurately at inference time, and those who want to train small, deployable VLM agents that approach that accuracy without a large model's cost. The Qwen3.5-4B result is aimed at the second group directly, since reinforcement learning inside the DocAtlas harness closed much of the gap to a far larger model's performance.

How to use it

The paper does not mention a code or dataset release, a license, or a price. The harness is described as a research system, tested with GPT-5.4 as the driving model at inference time and with Qwen3.5-4B as the model trained end-to-end via reinforcement learning inside the harness. Both usage modes draw on the same set of harness tools: search, reading, note-taking, and review.

How solid is it

The headline results come from one benchmark, MMLongBench-Doc: 71.4% for DocAtlas with GPT-5.4, above the benchmark's stated human-expert reference of 65.8%, and 63.7% for the RL-trained Qwen3.5-4B agent against a 54.4% baseline for the same model without the harness. The abstract names no authors or institutions, states no publication venue or conference, and gives no definition or size for 'a large margin' beyond those percentage figures.

Risks and caveats

The results rest on a single benchmark, MMLongBench-Doc, so how the harness design generalizes to other long-document tasks is untested here. No code or dataset release is mentioned, so the harness's implementation details, and any way to reproduce or independently verify the reported numbers, are not available from the abstract.