SKT trains AI agents to use skills via verified data

A team of 14 researchers, led by first author Zelin Tan and closing with Lei Bai, introduces SKT (Skill-Use Training), a pipeline for generating verified training data that teaches language-model agents to find, apply and combine reusable Agent Skills. The paper's motivation is that simply handing a model a library of skills does not mean it uses them well: identifying the right skill, applying it correctly and coordinating several skills together are separate capabilities that current models do not reliably have, even as the paper notes more than 600,000 public Agent Skills have appeared since October 2025 and that number keeps growing.

SKT works by selecting single-skill and multi-skill task configurations from a skills library, generating candidate tasks and executable trajectories, and verifying them through rule-based and agent-based checks with feedback-guided repair, keeping only trajectories that substantially use every skill a task requires. Applied to 2,000 distinct skills drawn from the skills.sh library, the pipeline produced 4,000 accepted task packages, split into 1,520 single-skill, 1,295 two-skill and 1,185 three-skill tasks. Four teacher models, MiniMax-M2.5, GLM-5, Qwen3.5-397B-A17B and DeepSeek V4 Pro, were used to attempt trajectory generation across 32,000 candidate task-teacher-harness combinations; verification retained 27,164 trajectories in total, 14,277 collected with the DeepAgents harness and 12,887 with OpenCode. The same pipeline, run against a separate held-out pool of skills, also produced SkillEval, a new executable benchmark for measuring skill use.

The authors fine-tuned two open models, Qwen3.5-9B and Gemma 4 E4B-IT, on the SKT trajectories and evaluated them across two agent harnesses and four benchmarks, for 16 model-harness-benchmark comparisons in total. SKT-based fine-tuning improved the mean score in every one of the 16 comparisons, with absolute gains on a 0-100 scale ranging from 3.20 to 18.91 points. The largest single gain came from Qwen3.5-9B on the DeepAgents harness evaluated on SkillEval, rising from 51.62 to 70.53; the smallest came from Gemma 4 E4B-IT on OpenCode evaluated on SkillsBench, rising from 7.08 to 10.28.

An ablation isolates what verification itself contributes: fine-tuning on trajectories from an unverified, unrepaired version of the same synthesis pipeline actually lowered all four benchmark means tested, while SKT's verified trajectories raised all four, leaving gaps of 11.91 to 24.61 points between the two approaches. A separate scaling experiment shows the benefit grows with the size of the training skill pool: for Qwen3.5-9B on OpenCode, SkillEval scores rise monotonically from 55.24 for the untrained baseline to 72.48 as the pool expands from 100 to 2,000 skills. Breaking results down by how many skills a task requires shows the gains are not limited to simple, single-skill tasks: SkillsBench improvements were 7.40 points for tasks needing one skill (25 tasks), 16.77 points for two skills (19 tasks), and 8.04 points for three or more skills (33 tasks).

Two further findings bound how the gains transfer. First, having the correct skill files available at inference time matters a great deal: gains fell to 0.53 to 5.69 points when the relevant skills were withheld from the model at test time, versus 8.68 to 18.91 points when they were provided. Second, training generalizes reasonably well across agent harnesses: for Qwen3.5-9B, a model trained on one harness and evaluated on the other retained 49.1% to 58.1% of the improvement seen when training and evaluation harnesses matched, an absolute gain of 4.35 to 9.86 points across four settings. A single checkpoint trained on a mixture of trajectories from both harnesses came close to matching harness-specific specialist checkpoints, differing by at most 2.71 points and outperforming the specialist in three of eight comparisons.

The paper does not disclose the model sizes of the four teacher models beyond their names (though Qwen3.5-397B-A17B's name implies a 397B-parameter model with a 17B active configuration), does not report training compute cost, and does not name any company or university affiliation for the authors. It is an arXiv preprint, submitted August 3, 2026, and has not been reviewed or accepted at a venue; no release date, license or public-availability statement is given for the SKT code, dataset or SkillEval benchmark.

Key facts

SKT synthesizes 4,000 verified task packages and 27,164 execution trajectories from 2,000 public agent skills drawn from the skills.sh library, using four teacher models: MiniMax-M2.5, GLM-5, Qwen3.5-397B-A17B and DeepSeek V4 Pro.
Fine-tuning Qwen3.5-9B and Gemma 4 E4B-IT on SKT trajectories improved scores in all 16 tested model-harness-benchmark combinations, with absolute gains ranging from 3.20 to 18.91 points on a 0-100 scale.
An ablation shows verification is the key ingredient: an unverified version of the same pipeline lowered all four benchmark means, while SKT raised all four, a gap of 11.91 to 24.61 points.
Gains scale with the training skill pool: for Qwen3.5-9B on the OpenCode harness, SkillEval score climbs monotonically from 55.24 for the untrained baseline to 72.48 as the pool grows from 100 to 2,000 skills.
Performance gains depend heavily on models having the relevant skill files available at inference: only 0.53 to 5.69 points when skills are withheld, versus 8.68 to 18.91 points when provided.

Why it matters

Agent Skills have exploded in number, with the paper citing more than 600,000 public skills available since October 2025, but a growing library does not translate into a model that reliably picks the right skill, applies it correctly, and coordinates several skills on one task. SKT addresses that gap directly by generating training data specifically for skill use rather than general task-solving, and its central finding is that verification, not raw data volume, is what makes the fine-tuning actually help: an unverified version of the same pipeline made models worse, while the verified version made every tested configuration better.

Who it affects

The work targets teams building or fine-tuning agents that rely on external skill libraries, whether they are training open models like Qwen or Gemma variants, building agent harnesses comparable to DeepAgents or OpenCode, or designing benchmarks to measure how well an agent uses tools and skills rather than just answers questions. It is most directly relevant to researchers and engineers working on the training-data side of agent development rather than to end users of any single product.

How to use it

The paper describes a training pipeline and a new benchmark, SkillEval, rather than a shipped product. It gives no release date, license, or public-availability statement for the SKT code, the training dataset, or SkillEval, so at the time of this paper there is no confirmed way for outside teams to obtain or run any of the three.

How solid is it

The results rest on a fairly broad evaluation: 2 fine-tuned models times 2 agent harnesses times 4 benchmarks, 16 comparisons in total, with improvement in every one, plus a dedicated ablation confirming that verification specifically (not just more synthetic data) drives the gains, and a scaling experiment showing the benefit grows smoothly as more skills are added to training. That said, this is an arXiv preprint that has not been peer-reviewed, one of the four benchmarks (SkillEval) was built by the same team using the same pipeline being evaluated, and no author affiliation is given in the visible text.

Risks and caveats

The paper omits training compute cost and any dollar figure for building the dataset or running the experiments, and does not name the organization behind the work. The reported benefit is also conditional: gains shrink sharply (to as little as 0.53 points) when a model does not have the relevant skill files available at inference time, meaning real-world deployment value depends on the right skills actually being present and discoverable at runtime, not just on the fine-tuning itself.