WebHarbor releases 153 dockerized website environments for training GUI agents
GUI web agents need environments that are realistic and interactive, and that stay stable through both evaluation and training. The authors of WebHarbor argue that current options fall short on both counts. Live website evaluation suffers from changing content, access restrictions and limited coverage of authenticated workflows. Existing synthetic training environments often lack functional depth and visual fidelity.
Their answer is WebHarbor, a suite of 153 executable, dockerized website environments. Each one is meant to faithfully reconstruct the visual interface, content and functional workflows of a real-world web service, and to run under deterministic control. The suite comes with 3,518 tasks. These span multi-step and post-authentication workflows, which the authors say existing works often fail to cover.
The environments are built through task-driven human-agent collaboration. An environment evolves as new tasks expand its functional scope. Coding agents implement the required changes, and human contributors audit visual fidelity, functional correctness and task validity.
Beyond evaluation, the authors present WebHarbor as a source of high-fidelity synthetic web environments for cost-efficient reinforcement learning (RL). As a demonstration, they report that a Qwen 4B-level GUI agent trained with RL in WebHarbor generalizes to unseen tasks and performs competitively with leading open-source GUI web agents across four benchmarks. The code and data are publicly available in the aiming-lab/WebHarbor repository on GitHub.
Key facts
- WebHarbor is a suite of 153 executable, dockerized website environments that reconstruct real-world web services under deterministic control.
- It includes 3,518 tasks covering multi-step and post-authentication workflows.
- Environments are built by coding agents and audited by human contributors for visual fidelity, functional correctness and task validity.
- The authors report a Qwen 4B-level GUI agent trained with RL in WebHarbor generalizes to unseen tasks and is competitive with leading open-source GUI web agents across four benchmarks.
- Code and data are public at github.com/aiming-lab/WebHarbor.
Why it matters
Testing web agents on live sites is unstable: content changes, access is restricted, and logged-in workflows are poorly covered. The authors say synthetic training environments, meanwhile, often lack functional depth and visual fidelity. WebHarbor aims to give one stable, deterministic setting that serves both evaluation and RL training, with tasks that reach multi-step and post-authentication flows.
Who it affects
Researchers and developers building and evaluating GUI web agents, particularly those who want to train with reinforcement learning without depending on live websites. The agent in the demonstration is small (Qwen 4B-level), so the work speaks to teams without large compute budgets.
How to use it
The code and data are publicly available at https://github.com/aiming-lab/WebHarbor. The environments are dockerized and executable, so they can serve as a fixed test bed for evaluation or as a training ground for RL.
How solid is it
The claims come from the paper's abstract and are the authors' own. The source gives no benchmark names or scores, so "competitive" is not quantified, and it does not say the agent outperforms the leading open-source agents. It also does not state which RL algorithm or exact Qwen model version was used.
Risks and caveats
The source does not name the real websites reconstructed or the categories of services. It gives no cost figures for the claimed cost-efficiency of RL, and no number of human contributors or amount of human effort. Faithfulness to real sites rests on the authors' own audits by human contributors, and generalization is reported on unseen tasks and four benchmarks that the source does not identify.