Glonce

SWE-Trail uses past runs to guide coding-agent training data collection

researchThe Glonce editorsHugging Face🔥57Glonce rating57Impact45Novelty60Relevance95Surprise25Players10Credibility72
SWE-Trail uses past runs to guide coding-agent training data collection

Coding agents solve repository issues through long sequences of exploration, editing and validation. Those trajectories are increasingly used as post-training data, so the way they are collected shapes what the trained model learns. The common approach is to sample several trajectories independently for each issue, then select or curate them. The paper's authors point out the weakness: as more trajectories are sampled independently, later ones become increasingly redundant, even though earlier ones could show where further collection effort is best spent.

The obvious fix is to show that history to the coding agent as advice. The authors say this backfires. Training on such trajectories can make the model rely on advice that is unavailable at test time, which creates a train-test mismatch. Their alternative is to let history guide where collection effort goes, while the coding agent alone decides what to generate.

That is SWE-Trail, a trajectory collector that uses earlier trajectories to steer later collection while keeping the history outside the agent's context. During a new rollout, SWE-Trail uses the history to pick a state where extra collection effort may pay off. At that point the coding agent proposes candidate next actions from its ordinary context. SWE-Trail then briefly executes those candidates from the same checkpoint to decide which route continues.

The authors judge collection approaches by training student models on the trajectories each one produces. For a 124B student initialized from Ling-3.0-flash-base, SWE-Trail improves mean resolution over independent sampling by 2.33 percentage points on SWE-bench Multilingual and 1.60 percentage points on SWE-bench Pro. They also test the same pipeline without history. It produces more resolved trajectories than independent sampling, but improves by only 0.11 points on Multilingual. The authors conclude that the results show the value of using history to guide collection without exposing it to the coding agent.

Key facts

  • SWE-Trail is a trajectory collector for coding-agent training data that uses earlier trajectories to guide where later collection effort goes.
  • The history stays outside the coding agent's context, which the authors say avoids a train-test mismatch caused by advice that is unavailable at test time.
  • At a chosen state, the agent proposes candidate next actions and SWE-Trail briefly executes them from the same checkpoint to decide which route continues.
  • For a 124B student initialized from Ling-3.0-flash-base, mean resolution rises by 2.33 points on SWE-bench Multilingual and 1.60 points on SWE-bench Pro versus independent sampling.
  • The same pipeline without history gains only 0.11 points on Multilingual.

Why it matters

Agent trajectories are increasingly used as post-training data, so how they are gathered shapes what the model learns. The paper targets a specific waste in the usual method: independent sampling per issue yields more and more redundant runs. Its twist is to use history to decide where to spend collection effort, not to coach the agent.

Who it affects

Teams that build post-training datasets for coding agents, and researchers working on SWE-bench-style issue resolution. The evaluation uses a 124B student initialized from Ling-3.0-flash-base.

How to use it

The text describes the method and results only. It does not mention a code, dataset or model release. The practical idea to take away: keep collection history out of the agent's context, use it to pick states worth extra effort, and test candidate actions with short executions from the same checkpoint.

How solid is it

The gains are 2.33 percentage points on SWE-bench Multilingual and 1.60 on SWE-bench Pro, both absolute differences in mean resolution over independent sampling, measured on one 124B student. The source gives no absolute resolution rates, no number of runs and no significance testing. The no-history ablation (0.11 points on Multilingual) supports the claim that history is what drives the gain on that benchmark. The no-history result on SWE-bench Pro is not stated. This is a preprint summary, not a peer-reviewed result.

Risks and caveats

The improvements are modest, and the evidence is one student model of one size. No trajectory counts, compute cost or collection budget are given, so the cost of the extra short executions is unknown. The claim about train-test mismatch from exposing history is the authors' argument. The source does not say how it was tested.