AllSpark's Iris-mini and Iris-pro top open-weight search agent benchmarks

Chinese lab AllSpark has released Iris-mini and Iris-pro, two open-source search agents built on Qwen-series base models, together with an evaluation harness and a paper describing how they were trained. Iris-mini has 35 billion parameters and builds on Qwen3.6-35B-A3B; Iris-pro has 397 billion parameters and builds on Qwen3.5-397B-A17B. Both work with a 256,000-token context window.

The training pipeline builds tasks backward from the link structure of web pages: starting from a seed page and its outgoing links, it constructs a graph of terms and relationships, then generates a multi-step question whose answer requires chaining several connected steps together. Every term except the final answer is replaced with a paraphrase, so the question cannot be resolved with a simple text search, and only questions a reference model cannot solve without tools but can solve with the right sources make it into the dataset. A stronger teacher model then generates solution paths made of reasoning, search queries and results, which pass through two rounds of filtering: one checking the full path for correctness, repetition loops and search depth, and a second, step-by-step review by a judge model whose criteria were derived from the data itself. From there the model improves through reinforcement learning against a live web search, with supervised fine-tuning and reinforcement learning alternating in a process the authors call "SFT-RL climbing." The judge model and result summaries run inside AllSpark's own training cluster on the team's own large Qwen model, so training does not depend on external services.

The team argues that runtime context management on common benchmarks often makes a bigger difference than the reported gaps between systems, since long research sessions can fill the context before an agent resolves all sub-questions. To isolate that effect, they tested every benchmark with and without context management while holding tools, context limits and the judge model constant, and the Iris scores come from a single agent with no helper agents or extra verification steps.

Across four benchmarks, with context management turned on, Iris-mini scores 82.2 on BrowseComp, 84.8 on BrowseComp-ZH, 86.9 on DeepSearchQA and 52.3 on Humanity's Last Exam. Iris-pro scores 88.6, 85.1, 92.9 and 56.4 on the same four tests. In the smaller class, Iris-mini leads on three of four benchmarks and beats the next-best model, XYZ-Aquila-mini, on BrowseComp by 3.4 points, though it trails on DeepSearchQA. Iris-pro leads or ties in the larger class. Context management has a far bigger effect on the smaller model, boosting its BrowseComp score by up to 21.2 points, which the paper attributes to faster context consumption rather than a smaller token budget: Iris-mini needs more steps for the same tasks and hits the context limit more often. Gains on Humanity's Last Exam are smaller because that benchmark leans more on domain knowledge than web search.

In the paper's appendix, the team describes a BrowseComp-ZH question about the TV series "Game of Thrones" where the agent answered "Bolton" but the recorded ground truth said "Lannister"; since the character Sansa Stark's second marriage is to Ramsay Bolton, the agent's answer was correct and the ground truth was wrong. The team cites cases like this as motivation to build better benchmarks. Beyond search, the authors report that the training data and the resulting models also improved performance on tasks they were never trained for, including general tool use and office work, suggesting search may function as a foundational skill rather than a narrow specialty.

The model weights are available in a Hugging Face collection and the code is on GitHub. The current release includes the Iris Harness with the agent loop, tools, context management strategies and all four benchmarks with evaluation code; it runs against any OpenAI-compatible endpoint. The team says it plans to release the data construction and training pipelines later.

Key facts

• AllSpark released Iris-mini (35 billion parameters, on Qwen3.6-35B-A3B) and Iris-pro (397 billion parameters, on Qwen3.5-397B-A17B), both with a 256,000-token context window.
• Training questions are reverse-engineered backward from the link structure of web pages, then paraphrased so answers cannot be found by simple text search.
• With context management on, Iris-mini scores 82.2/84.8/86.9/52.3 and Iris-pro scores 88.6/85.1/92.9/56.4 on BrowseComp, BrowseComp-ZH, DeepSearchQA and Humanity's Last Exam.
• Context management alone boosts Iris-mini's BrowseComp score by up to 21.2 points, more than it helps the larger Iris-pro.
• Weights are on Hugging Face and the evaluation harness is on GitHub now; the data construction and training pipelines are promised for later.

Why it matters

Most search-agent papers report a final score; this one publishes a method for building the training data itself, generated backward from the link structure of real web pages rather than hand-written by people. It also isolates a confound that inflates many leaderboard numbers: how much of an agent's score comes from the model versus from runtime tricks like discarding context, which the team strips out by testing every benchmark both ways.

Who it affects

Anyone building or evaluating search and research agents on top of open-weight models gets a released harness, benchmark suite and a documented training recipe to compare against or reuse, rather than a closed leaderboard entry. The reported side effect, that search training also lifted general tool use and office-task performance, matters to teams building broader agents, not just search ones.

How to use it

The model weights for Iris-mini and Iris-pro are available in a Hugging Face collection, and the code is on GitHub. The current release covers the Iris Harness (agent loop, tools, context management strategies, and all four benchmarks with evaluation), which runs against any OpenAI-compatible endpoint. The data construction and training pipelines are not out yet; the team says those come later.

How solid is it

The benchmark numbers are self-reported by AllSpark in its own paper, run through a judge model the team also built and derived from its own data. The comparison to competing models rests on a single stated margin, 3.4 points over XYZ-Aquila-mini on BrowseComp, with no other competitor scores given. The team's own appendix anecdote, an agent penalized for a correct answer because a benchmark's ground truth was wrong, cuts both ways: it shows the team scrutinizing its own evaluation, but also that at least one of the four benchmarks used to rank these models contains verified errors.

Risks and caveats

The paper names no individual authors, only "the team," and gives no release date for the models or the paper itself. The promised data-construction and training pipelines have no announced timeline, so outside labs cannot yet reproduce the training method, only run the released harness against the finished weights. As with any self-reported benchmark suite, independent replication on held-out tasks has not happened yet.