OS-Helix agent merges GUI and code actions, reports 66.44% on OSWorld-Verified
The OS-Helix paper starts from a practical observation: real desktop automation is neither purely visual nor purely programmatic. An agent has to understand user-facing interfaces while also handling files, scripts and system state efficiently. The authors say that this mix cannot be reached by composing tools at the prompt level. In their preliminary experiments, simply giving a GUI agent a code tool made it worse. From that they conclude that the choice between GUI and code must be learned through training on high-quality hybrid data.
OS-Helix is their answer: an end-to-end trained multimodal computer-using agent that puts GUI actions and code execution into a single action space.
The training data is the hard part, and the paper describes two pieces. The first is application-artifact composition, which pairs desktop applications with authentic file artifacts to instantiate executable Ubuntu states. The second is a decoupled instruction/evaluator generation pipeline. It produces tasks grounded in the environment, and their final states can be verified whatever way the task was solved, by clicking or by code.
Training runs in two stages. The policy is cold-started with supervised fine-tuning on filtered hybrid data, then improved with online reinforcement learning in live desktop environments. For the RL stage the authors propose Step-balanced GSPO. It aggregates importance weight by action step instead of by raw assistant tokens, so that long code actions do not dominate optimization when GUI actions and code actions are mixed.
On OSWorld-Verified, OS-Helix-27B and OS-Helix-35B-A3B reach relative improvements of 35.0% and 33.7% over their matched Qwen3.5 hybrid baselines. OS-Helix-27B-RL then improves a further 42.9% and reaches 66.44% success. The abstract does not state the reference point for the 42.9% figure, nor whether it is relative or absolute. Ablations, the authors say, give evidence consistent with contributions from learned GUI/code coordination, scalable verifiable supervision and action-step-balanced optimization. They also report cross-platform transfer on WindowsAgentArena. The project homepage is at https://os-copilot.github.io/OS-Helix/.
Key facts
- OS-Helix is an end-to-end trained multimodal computer-using agent that unifies GUI actions and code execution in one action space.
- The authors report that naively adding a code tool to a GUI agent degraded it in preliminary experiments, and argue GUI/code routing must be learned from hybrid data.
- Training uses SFT on filtered hybrid data, then online RL in live desktop environments with a proposed Step-balanced GSPO, which weights by action step rather than raw assistant tokens.
- On OSWorld-Verified, OS-Helix-27B and OS-Helix-35B-A3B gain 35.0% and 33.7% (relative) over matched Qwen3.5 hybrid baselines.
- OS-Helix-27B-RL reaches 66.44% success on OSWorld-Verified after a further 42.9% improvement.
Why it matters
Real desktop automation is neither purely visual nor purely programmatic: tasks often call for scripts and file manipulation as well as clicking through interfaces. The paper's central claim is that bolting a code tool onto a GUI agent through prompting does not work and can hurt, so the skill of picking between GUI and code has to be trained in. OS-Helix is an attempt to do that with one action space, verifiable tasks built from real applications and files, and an RL objective adapted to mixed short and long actions.
Who it affects
The work is aimed at people building and training desktop automation agents, particularly multimodal models. The reported results are for 27B and 35B-A3B models built against Qwen3.5 hybrid baselines, so teams working with models of that class are the nearest audience.
How to use it
The abstract points to a project homepage at https://os-copilot.github.io/OS-Helix/. No release or licensing of weights or code is stated in the text, so practical reuse depends on what that page offers. The ideas described are the application-artifact composition for building executable Ubuntu task states, the decoupled instruction/evaluator pipeline for verifiable tasks, and Step-balanced GSPO for RL on mixed GUI and code actions.
How solid is it
This is a preprint listed on Hugging Face Papers, and everything here comes from its abstract. The headline figure, 66.44% success for OS-Helix-27B-RL on OSWorld-Verified, is the authors' own report. The gains over the Qwen3.5 hybrid baselines are given only as relative percentages. No absolute success rates are given for those baselines or for the pre-RL OS-Helix models. No comparison against other published agents or human performance is given. The authors describe their ablation results as evidence consistent with their claims, not proof.
Risks and caveats
The 42.9% figure has no stated reference point and is not labelled relative or absolute, so it should not be added to or combined with the 35.0% and 33.7% gains. No WindowsAgentArena numbers are given, so the cross-platform transfer claim cannot be sized from the abstract. No training data size or compute cost is stated. The claim that prompt-level tool composition cannot give native GUI/code coordination rests on the authors' preliminary experiments, which the abstract does not detail.
“native GUI/code coordination cannot be obtained by prompt-level tool composition”
— OS-Helix paper abstract