AutoMaster agent beats vanilla SFT by 1.42-8.99 points with Skill-FT

The paper starts from a simple observation: task success alone does not say which behaviors post-training should reinforce. Even successful teacher trajectories can contain unsupported assumptions, unproductive repetition, or superficial verification. Training a smaller model on all of them copies those habits along with the good outcomes.
The authors propose Skill-FT (Skill-Guided Fine-Tuning). It uses natural-language skills to suppress undesirable habits and to elicit effective behaviors that are underrepresented while the teacher generates its rollouts. A target model is then fine-tuned on the resulting trajectories.
On top of that they introduce AutoMaster, an autonomous post-training system. An LLM agent develops the interventions itself, working through trajectory analysis, skill design, data selection, and feedback from rollouts and from evaluation of the target model.
The headline result: across Terminal, Search, and software engineering tasks, with 4B and 9B target models, AutoMaster improves mean performance by 1.42-8.99 percentage points over Vanilla SFT. The baseline is SFT on successful unguided teacher trajectories, and the teacher, task pool, training set size, and SFT recipe are held fixed. Together with ablations and trajectory analyses, the authors say these results suggest that behavioral shaping can make a fixed model a more effective teacher.
They also report a second finding. Keeping the skill instructions in the training examples during SFT gave higher mean performance than removing them, in all six settings. This held even though the instructions are absent at evaluation and the supervised outputs are identical. The authors say this suggests that preserving skill context as part of a training demonstration can support better behavioral internalization.
Key facts
- Skill-FT uses natural-language skills to suppress undesirable habits and elicit underrepresented effective behaviors during teacher rollouts, then fine-tunes a target model on those trajectories.
- AutoMaster is an autonomous post-training system in which an LLM agent handles trajectory analysis, skill design, data selection, and feedback from rollouts and target-model evaluation.
- Across Terminal, Search, and software engineering tasks with 4B and 9B target models, mean performance rises by 1.42-8.99 percentage points over Vanilla SFT.
- Keeping skill instructions in the SFT data beat removing them in all six settings, even though the instructions are absent at evaluation and the supervised outputs are identical.
Why it matters
Distilling a stronger model into a smaller one usually keeps only trajectories that succeeded. The paper argues that success is a weak filter, because successful runs can still carry unsupported assumptions, repetition, or shallow verification. Its answer is to shape how the teacher behaves before the data is collected, rather than only choosing which outputs to keep. It also hands that shaping work to an LLM agent. The authors read their results as a sign that a fixed model can be made a better teacher through behavioral shaping.
Who it affects
The work concerns people who fine-tune smaller models on teacher-generated trajectories for agentic tasks such as terminal use, search, and software engineering. The experiments use 4B and 9B target models.
How to use it
The abstract describes the method but not a recipe to run. The loop it outlines: analyze teacher trajectories, write natural-language skills that discourage bad habits and encourage underrepresented good ones, select data, and check the result against rollouts and target-model evaluation. One practical detail from the paper: leave the skill instructions in the training examples during SFT rather than stripping them out. The source mentions no code, model or dataset release.
How solid is it
This is a preprint listed on Hugging Face Papers, and the source is its abstract. The comparison is controlled: teacher, task pool, training set size, and SFT recipe are held fixed against Vanilla SFT on successful unguided teacher trajectories. The authors word their conclusions as suggestions, not proof. The source does not say which setting produced the 1.42 gain and which the 8.99 gain. It also gives no benchmark names or absolute scores. The teacher model and the 4B and 9B target models are not named, and neither is the LLM agent that powers AutoMaster.
Risks and caveats
The gain range is wide, from 1.42 to 8.99 percentage points, so the effect varies a lot by setting. The abstract gives no compute cost or training time for running the agent. No authors or institutions are named in the source. The claim about retaining skill instructions comes from six settings, and the authors call it a suggestion about behavioral internalization, not a demonstrated mechanism.
“Task success alone does not specify which behaviors post-training should reinforce.”
— Paper abstract