Glonce

Player-1 trains computer-use game agents with RL on 16 synthetic environments

researchThe Glonce editorsHugging Face🔥59Glonce rating59Impact45Novelty60Relevance95Surprise35Players15Credibility75
Player-1 trains computer-use game agents with RL on 16 synthetic environments

A paper on Hugging Face introduces Player-1, a post-training framework for teaching computer-use models to control video games. The authors start from a familiar problem: computer-use agents already read screens and issue keyboard and mouse actions well, but games, with their visually rich worlds and varied mechanics, remain hard. Current game-agent pipelines often lean on large-scale demonstrations that cover dynamic game states poorly, and scalable environments for interactive training are difficult to obtain.

Player-1 answers with a two-stage recipe. First, the authors extend the computer-use action space with relative pointer motion, sustained inputs and composite actions. Then comes supervised initialization: computer-use models are fine-tuned on 6,044 curated gameplay trajectories that pair recorded actions with task instructions and visual reasoning.

Second comes reinforcement learning. The authors build Player-1-Gym, a suite of sixteen synthetic 2D and 3D environments designed around control demands shared across games. Programmatically verifiable rewards, plus randomized observations and control dynamics, support multi-environment reinforcement learning with PPO.

The resulting models, Player-1-9B and Player-1-35B-A3B, improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points respectively. Player-1-9B reaches 38.33% success compared with 12.63% for the strongest evaluated baseline, and the paper reports further gains over SFT on ViZDoom-10 and GameWorld.

Two analyses are highlighted. Broader synthetic environment coverage improves transfer, and the use of composite actions that combine multiple keyboard and mouse inputs increases during RL. The authors conclude that learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite.

Key facts

  • Player-1 is a post-training framework: supervised fine-tuning on 6,044 curated gameplay trajectories, then multi-environment reinforcement learning with PPO.
  • Player-1-Gym consists of sixteen synthetic 2D and 3D environments built around control demands shared across games, with programmatically verifiable rewards and randomized observations and control dynamics.
  • Player-1-9B and Player-1-35B-A3B improve MineDojo-300 success over their SFT policies by 27.8 and 17.7 percentage points respectively.
  • Player-1-9B reaches 38.33% success against 12.63% for the strongest evaluated baseline, with further gains over SFT on ViZDoom-10 and GameWorld.
  • The computer-use action space is extended with relative pointer motion, sustained inputs and composite actions, and composite action use grows during RL.

Why it matters

Games are a demanding test of general digital control, and the paper argues that demonstration-heavy pipelines cover dynamic game states poorly. Player-1 tests a different route: synthetic environments with checkable rewards, used for reinforcement learning on top of a modest supervised start of 6,044 trajectories. The reported result is that this RL stage adds large gains over the SFT policy, and that the gains carry beyond the environments used for RL training.

Who it affects

The work is aimed at researchers building computer-use and game-playing agents, especially those who depend on large demonstration sets or struggle to obtain interactive training environments. It is also relevant to anyone studying whether synthetic environments can substitute for real game data in RL post-training.

How to use it

This is a research preprint, not a product. The abstract describes the recipe: extend the action space with relative pointer motion, sustained inputs and composite actions; fine-tune on curated trajectories pairing actions with instructions and visual reasoning; then run multi-environment PPO in Player-1-Gym with verifiable rewards and randomized observations and dynamics. No code, weights or dataset release is mentioned.

How solid is it

The evidence is the paper's own reported results: MineDojo-300 gains of 27.8 and 17.7 percentage points over SFT, a 38.33% versus 12.63% comparison against the strongest evaluated baseline, and further gains on ViZDoom-10 and GameWorld. The abstract names no authors or institutions, and the names of the baseline models, including the strongest evaluated baseline, are not given. No figures are given for the ViZDoom-10 and GameWorld gains, and the base models that were fine-tuned are not named.

Risks and caveats

The claims come from an abstract, so the size of the baselines, the evaluation setup and the SFT starting scores cannot be checked from it; the abstract does not state the SFT baseline success rates in absolute terms. The strongest baseline is unnamed, which limits how far the 38.33% versus 12.63% comparison can be interpreted. Generalization is claimed beyond the RL training suite, but only the benchmarks named above are reported.

“learning from synthetic experience substantially improves game control beyond supervised initialization, with gains that generalize beyond the RL training suite”

— Player-1 paper abstract