SkillForge distills project-specific skills for coding agents from synthetic issues
Large language model based coding agents already show strong general ability at automated software issue resolution, but they often struggle once the task is a specific repository, because they lack project-specific knowledge about that codebase. Existing self-evolving methods have tried to supply that missing knowledge from a repository's history or from repair trajectories generated through online exploration, but each route carries a cost: relying on history only works where enough historical issue-resolution signal already exists, and generating repair trajectories through live exploration means paying substantial computation for every single new issue.
To address this, researchers propose SkillForge, a self-distillation framework that proactively acquires project-specific knowledge straight from the repository itself, rather than waiting for a real issue to expose a gap in that knowledge. Instead of depending on a repository's issue history or on live exploration, SkillForge synthesizes its own project-specific issues by re-implementing core functionality in the repository that already has test coverage. By resolving these self-generated issues, SkillForge distills what it learns into reusable, entity-grounded skills and associates each skill with the relevant repository entities, so that project-specific knowledge is already on hand when a real issue involving those entities comes in later.
The paper reports extensive experiments using both open-source and closed-source models, and states that SkillForge consistently improves issue-resolution performance over strong baselines. It does not, however, give the actual accuracy or resolve-rate figures behind that claim, nor does it name the benchmark, dataset, repositories, baselines, or specific models used in testing. The authors conclude that proactively acquiring project-specific knowledge before a real issue arrives substantially improves downstream software issue resolution, compared with only acquiring that knowledge once an issue is already in hand.
Key facts
- LLM coding agents already handle general automated issue resolution well, but they often fail on issues specific to one repository because they lack project-specific knowledge of that codebase.
- Existing self-evolving approaches try to supply that knowledge from a repository's issue history or from online repair trajectories, but the first needs enough historical signal to already exist and the second costs substantial computation per issue.
- SkillForge instead proactively synthesizes its own project-specific issues by re-implementing core, test-covered functionality already in the repository, rather than waiting for a real issue to expose the gap.
- Resolving those synthetic issues lets SkillForge distill reusable, entity-grounded skills tied to specific repository entities, so the knowledge is ready before a matching real issue arrives.
- Experiments with open-source and closed-source models show SkillForge consistently improves issue-resolution performance over strong baselines, though the paper names no specific figures, benchmarks, baselines, or models.
Why it matters
AI coding agents already do well at automated issue resolution in general, but the practical bottleneck is applying that skill to one specific, unfamiliar repository, where an agent lacks project-specific knowledge. The two existing fixes both carry a cost: leaning on a repository's history of resolved issues only works once that history exists, and generating repair trajectories through live exploration is expensive to redo for every new issue. SkillForge's contribution is to acquire that project-specific knowledge proactively, straight from the repository's own test-covered code, before any real issue exists, which the authors say substantially improves how well an agent later resolves real issues.
Who it affects
The paper's stated motivation centers on LLM based coding agents meant to operate against one specific, real codebase rather than generic benchmark problems, since that is exactly where general-purpose issue-resolution skill falls short. That makes SkillForge relevant to work on self-evolving or self-improving coding agents, and to anyone who might rely on such agents to resolve issues in their own repositories. The text does not name any particular companies, products, or repositories involved in the work.
How to use it
SkillForge is presented as a method rather than a shipped product: the text gives no pricing or licensing terms and no statement on whether code, trained skills, or data are released. Conceptually, applying it means pointing the framework at a target repository, letting it synthesize practice issues from functionality that already has test coverage, running an agent through those synthetic issues, and keeping the resulting entity-grounded skills on hand for when a real issue later touches the same parts of the code.
How solid is it
The evidence offered is extensive experiments across open-source and closed-source models, with SkillForge said to consistently beat strong baselines on issue resolution. The text gives none of the specifics needed to weigh that claim further: no accuracy or resolve-rate figures, no named benchmark or dataset, and no named baselines or models. No author names, affiliations, or institutions appear anywhere in the paper's own text either; the single name attached to the hosting listing is metadata from that page, not a fact stated in the article itself.
Risks and caveats
The central claim, that SkillForge consistently and substantially improves issue resolution, is stated only in qualitative terms, with no figures, named benchmarks, named baselines, or named models given to size the improvement or let it be checked independently. The method's reach is also bounded by its own mechanism: because SkillForge synthesizes practice issues by re-implementing functionality that already has test coverage, it can only build project-specific skills around the parts of a repository that tests already cover.