Glonce

PARSER splits reading from reasoning in long-context LLM agents, cuts latency by up to 11x

agentsThe Glonce editorsHugging Face🔥76Glonce rating76Impact68Novelty78Relevance98Surprise62Players35Credibility92
PARSER splits reading from reasoning in long-context LLM agents, cuts latency by up to 11x

Most long-context LLM agents today read a document the way a person would: chunk by chunk, in order, updating a compact memory state as they go. That sequential design ties how deep an agent can reason to how far it has read, which creates two problems. First, the agent becomes sensitive to where the important evidence sits in the document, since reasoning that has already moved past a chunk gets less benefit from evidence buried there. Second, inference latency scales linearly with document length, so a longer document costs proportionally more time regardless of how much reasoning the task actually needs.

A new paper introduces PARSER, an architecture that decouples reading from reasoning. Instead of one agent walking through the document in order, a bank of lightweight subagents is used, each bound to a single chunk, and together they read the entire document in parallel. A separate lead agent handles the reasoning: it runs iterative scatter-gather rounds, broadcasting a query to all the subagents at once, collecting the evidence they return, and then formulating a deeper follow-up query based on what has been found so far. Reasoning depth is no longer tied to how much of the document has been read; the lead agent can run as many rounds as the task needs.

Only the lead agent is trainable in this setup. It is optimized with reinforcement learning, while the subagents are left frozen, off-the-shelf models that are not fine-tuned for the task.

On multi-hop question answering with document contexts ranging from 7K to 896K tokens, a PARSER built on a 4B backbone beats the strongest sequential memory baseline by 5.7 points on average across that range, and by 12.0 points specifically at the 896K-token end. Scaled up to a 9B backbone, PARSER surpasses DeepSeek-V4-Pro by 6.3 points.

Controlled experiments show PARSER stays robust when the position, order or distance of the evidence within the document is perturbed, conditions that cause large accuracy swings for sequential methods. On top of that robustness, the parallel-reading design cuts inference latency by up to 11x compared with those sequential approaches.

Key facts

  • PARSER decouples reading from reasoning in long-context LLM agents: a bank of subagents, each bound to one chunk, reads the whole document in parallel, while a separate lead agent reasons through iterative scatter-gather query rounds.
  • Only the lead agent is trained, using reinforcement learning; the subagents stay frozen, off-the-shelf models.
  • On multi-hop QA with contexts from 7K to 896K tokens, a 4B-backbone PARSER beats the strongest sequential memory baseline by 5.7 points on average and by 12.0 points at 896K tokens.
  • Scaled to a 9B backbone, PARSER beats DeepSeek-V4-Pro by 6.3 points.
  • PARSER cuts inference latency by up to 11x versus sequential methods and stays robust when evidence position, order or distance is perturbed, conditions that badly hurt sequential approaches.

Why it matters

The bottleneck in long-context LLM agents has been structural: reading a document sequentially ties reasoning depth to reading progress, so the agent pays for document length whether or not the task needs that much reasoning, and it can be thrown off by evidence sitting in the wrong place in the text. PARSER's contribution is to break that coupling. Because a bank of subagents reads every chunk of the document in parallel, the lead agent's reasoning no longer has to wait on sequential traversal: it can run as many scatter-gather query rounds as the task requires, independent of how long the document is. That structural change is what produces the paper's two headline results: latency that does not scale with document length the way sequential reading does, and steadier accuracy when evidence is scattered, reordered, or placed far from where a sequential reader would expect it.

Who it affects

The paper is aimed at people building long-context LLM agents, specifically for multi-hop question answering over documents long enough that reading them sequentially becomes a bottleneck; the tests cover context lengths from 7K tokens up to 896K tokens. That includes engineers benchmarking their own long-context agent designs against sequential memory baselines and against systems such as DeepSeek-V4-Pro, since those are the comparisons the paper itself draws.

How to use it

PARSER is presented here as a research architecture rather than a released tool. What the text specifies is the design itself: split a long document across a bank of frozen, off-the-shelf subagents, each responsible for one chunk, and pair them with a separate lead agent trained with reinforcement learning to run scatter-gather query rounds over their combined evidence. Because only the lead agent needs training, the pattern is meant to sit on top of existing off-the-shelf models rather than requiring a new base model built from scratch. The paper gives no publication venue, timeline, or code and model release information, so there is nothing to install or reproduce beyond following the architecture as described.

How solid is it

The evidence comes from multi-hop QA tests spanning a wide range of context lengths, from 7K to 896K tokens, rather than a single fixed size. At a 4B backbone, PARSER is measured against what the paper calls the strongest sequential memory baseline, beating it by 5.7 points on average across that range and by 12.0 points at the 896K-token end. At a 9B backbone, it is measured against a named system, DeepSeek-V4-Pro, and beats it by 6.3 points. The paper also reports controlled experiments testing robustness to evidence position, order and distance specifically, rather than relying only on aggregate accuracy scores. The 4B-scale baseline itself is not named, which caps how far these particular numbers can be checked against the wider field.

Risks and caveats

Several gaps limit how far this can be checked from the text alone. The strongest sequential memory baseline that PARSER beats at 4B scale is never named, so there is no way to judge how strong that comparison point really is or how it was tuned. No benchmark or dataset name is given for the multi-hop QA tests, and no reinforcement learning algorithm is named for training the lead agent, which limits how precisely the setup could be reproduced. The up to 11x latency figure is also relative only, with no absolute reference point, such as seconds or tokens per second, attached to it.