Glonce

RST synthesizes 37,484 terminal-agent tasks at $0.05 each

researchThe Glonce editorsHugging Face🔥78Glonce rating78Impact72Novelty75Relevance100Surprise58Players72Credibility82
RST synthesizes 37,484 terminal-agent tasks at $0.05 each

High-quality long-horizon training data for terminal agents is expensive to hand-author, hundreds to thousands of dollars per task, because the instruction, environment, reference solution, and verifier all have to stay mutually consistent. The authors say human authoring does not scale, and that direct generation with large language models often breaks these dependencies. They present Recursive Synthetic Terminal Tasks (RST), a recursive verified synthesis framework for building long-horizon terminal-agent tasks at scale.

Starting from verified seed tasks, RST extends the reference solution, realigns the verifier and instruction to match the new workflow, validates the result in a fresh sandbox, and reuses each accepted task as a seed for the next round. Across fifteen recursive rounds, this pipeline produced 37,484 synthesized terminal-agent tasks at a cost of roughly $0.05 per task.

Task difficulty rises sharply across the rounds. The median reference solution grows from 67 lines at round 1 to 374 lines at round 15, and the median number of executed commands grows from 40 to 244 over the same span. As a check on that rising difficulty, DeepSeek-V4-Pro's pass@4 rate on the tasks falls from 90% at round 1 to 2.5% at round 15.

To demonstrate training utility, the researchers collected rejection-sampled trajectories from Qwen3.5 on the synthesized tasks and used them for supervised fine-tuning. That fine-tuning improved Qwen3.5-27B and Qwen3.5-122B-A10B by up to 10 points across three benchmarks: Terminal-Bench 2, Terminal-Bench Hard, and Long-Horizon Terminal Bench. Agentic PPO training pushed Qwen3.5-27B further, to 49.44%, 32.00%, and 22.07% on those three benchmarks respectively, relative gains of 20.0%, 41.2%, and 21.9% over the base model.

The authors report that after 15 rounds the recursion shows no ceiling: synthesis yield and validation rates stay stable even as task difficulty keeps climbing, which they say indicates the process can continue well beyond the scale reported in this work.

Key facts

  • RST recursively extends verified seed tasks over fifteen rounds, producing 37,484 long-horizon terminal-agent tasks at roughly $0.05 per task.
  • Task difficulty rises sharply: median reference solution length grows from 67 to 374 lines and median executed commands from 40 to 244 between round 1 and round 15.
  • DeepSeek-V4-Pro's pass@4 success rate on the synthesized tasks drops from 90% at round 1 to 2.5% at round 15 as difficulty increases.
  • Supervised fine-tuning on RST-generated trajectories improves Qwen3.5-27B and Qwen3.5-122B-A10B by up to 10 points on three terminal-agent benchmarks; agentic PPO lifts Qwen3.5-27B to 49.44%, 32.00%, and 22.07% on them, gains of 20.0%, 41.2%, and 21.9% over the base model.
  • After 15 rounds the authors report no ceiling: synthesis yield and validation rates stay stable as task difficulty keeps climbing.

Why it matters

High-quality long-horizon training data for terminal agents is expensive to hand-author, hundreds to thousands of dollars per task, because the instruction, environment, reference solution, and verifier all have to stay mutually consistent, and the authors say human authoring does not scale while direct LLM generation often breaks those dependencies. RST replaces manual writing and unchecked generation with a verified recursive process, producing tens of thousands of tasks at roughly $0.05 each.

Who it affects

Teams building or training terminal and coding agents, and researchers working on agentic benchmarks such as Terminal-Bench 2, Terminal-Bench Hard, and Long-Horizon Terminal Bench, since the synthesized tasks and the trajectories collected on them are what such training and evaluation pipelines consume.

How to use it

The paper describes RST as a synthesis pipeline: starting from verified seed tasks, it extends the reference solution, realigns the verifier and instruction to the new workflow, validates the result in a fresh sandbox, and reuses accepted tasks as seeds for the next round, across fifteen rounds in the reported run. The text does not state any code or dataset release terms, pricing, or licence.

How solid is it

The paper reports concrete, checkable outputs: 37,484 synthesized tasks, measured growth in reference-solution length (67 to 374 median lines) and executed commands (40 to 244 median) across rounds, and a measured drop in DeepSeek-V4-Pro's pass@4 from 90% to 2.5% as a difficulty check. Training utility is demonstrated with both supervised fine-tuning and agentic PPO on two Qwen3.5 model sizes, evaluated across three separate benchmarks, with gains reported for both methods.

Risks and caveats

The text does not name the authors' affiliations, gives only a per-task cost rather than a total for the full 37,484-task run, and does not describe what domain or environment the terminal tasks cover beyond "terminal-agent tasks." It also does not explain what specifically drives DeepSeek-V4-Pro's pass@4 decline beyond rising task difficulty, and the human-authoring cost figure, the training results, and the "no ceiling" finding are all self-reported by the paper's own authors rather than independently verified.