Glonce

WorkFactory builds 112,686 professional tasks to train WorkPilot agents

researchThe Glonce editorsHugging Face🔥58Glonce rating58Impact45Novelty60Relevance95Surprise30Players15Credibility75
WorkFactory builds 112,686 professional tasks to train WorkPilot agents

The paper starts from a bottleneck in training agents for professional work. Scaling supervised training needs a wide range of professional tasks that produce reliable expert demonstrations. But when a task is missing information or contains inconsistent information, the teacher model's execution can be obstructed, and the supervision drawn from it gets weaker.

The authors propose WorkFactory, a scalable framework for constructing coherent and executable professional tasks. It works in two stages. First, it starts from professional intents and does information-level planning: it builds an information support graph that guides how a task is instantiated. The graph explicitly represents the dependencies and locations of the information needed for the intended outputs. Second, probe executions run against the task to identify support failures and guide repairs, while preserving the intended work.

Using the framework, the authors construct 112,686 tasks spanning 612 occupations. They collect teacher trajectories on these tasks and use them to fine-tune Qwen models at 4B, 27B and 35B-A3B scales. The resulting models are the WorkPilot family.

According to the authors, all three WorkPilot models improve over their respective base models across four professional-work benchmarks. The one number given is for WorkPilot-35B-A3B, which goes from 879 to 1565 Elo on GDPval-AA v2.1. Scaling analyses show model-dependent diminishing returns, with execution length first increasing and then decreasing. Controlled component analyses show that planning and repair reduce probe-diagnosed task environment defects and improve downstream performance.

The data and models are available in a GitHub repository at github.com/IQuestLab/WorkFactory.

Key facts

  • WorkFactory builds executable professional tasks from professional intents, using an information support graph for planning and probe executions to find and repair information gaps.
  • The authors constructed 112,686 tasks spanning 612 occupations and used teacher trajectories to fine-tune Qwen models at 4B, 27B and 35B-A3B scales, called WorkPilot.
  • All three WorkPilot models improve over their base models on four professional-work benchmarks; WorkPilot-35B-A3B goes from 879 to 1565 Elo on GDPval-AA v2.1.
  • Component analyses show that planning and repair reduce probe-diagnosed task environment defects and improve downstream performance.
  • The data and models are released at github.com/IQuestLab/WorkFactory.

Why it matters

Training agents for professional work depends on demonstrations from a teacher model, and those demonstrations are only as good as the tasks behind them. The paper targets the point where tasks fail: missing or inconsistent information that blocks the teacher and weakens the supervision. Its answer is to plan what information a task needs and where it sits, then test the task with probe executions and repair it. The scale reported is 112,686 tasks across 612 occupations.

Who it affects

Mainly researchers and teams building and fine-tuning agents for professional work, who need large sets of executable tasks with reliable demonstrations. The released data and WorkPilot models may also be of use to anyone comparing results on professional-work benchmarks such as GDPval-AA v2.1.

How to use it

The authors state that the data and models are available at github.com/IQuestLab/WorkFactory. The abstract does not describe licence terms or usage instructions, so check the repository for those.

How solid is it

The claims are the authors' own, from the paper's abstract. The headline result is concrete: WorkPilot-35B-A3B improves from 879 to 1565 Elo on GDPval-AA v2.1, and all three models improve on four benchmarks. The authors also report controlled component analyses showing that planning and repair reduce task environment defects and improve downstream performance. Only the 35B-A3B Elo change is given numerically; the other three benchmarks are not named, and the results for the 4B and 27B models are not given as figures.

Risks and caveats

The Elo figures are absolute ratings on one benchmark and say nothing here about how the 879 base model compares with other models or with human experts. The abstract does not name the teacher model that produced the trajectories or the Qwen model version. The authors note model-dependent diminishing returns as training scales, with execution length first increasing and then decreasing, so more tasks do not help indefinitely; the abstract does not say at which sizes or at what point.