Glonce

Online RL trains 8B UI agents on live iOS apps, tying Gemini-3.1-Pro

researchThe Glonce editorsHugging Face🔥56Glonce rating56Impact42Novelty60Relevance95Surprise25Players12Credibility75
Online RL trains 8B UI agents on live iOS apps, tying Gemini-3.1-Pro

The paper starts from a trade-off in UI navigation. Frontier vision-language models are strong at it, but their high inference costs prohibit everyday use. Smaller models trained with supervised fine-tuning (SFT) or offline reinforcement learning often memorize static trajectories, which leads to brittle execution and an inability to recover from errors.

The authors propose an online reinforcement learning framework that trains UI navigation agents directly on live iOS applications. The agent is split into two specialized components: a Planning agent and a UI Grounding model.

Rewards come from an LLM-as-a-Judge. It combines dense, step-level feedback on intermediate progress with a terminal reward for overall task completion. To use both signals, the agent is optimized with Proximal Policy Optimization (PPO) with step-level advantage estimation.

The authors also introduce GiGPPO, which they call a novel neighbor-conditioned value estimation method. It aims to make the critic more stable by using reward signals from visually similar states across concurrent trajectories.

To run this online training at scale, they built infrastructure that orchestrates hundreds of concurrent virtual iOS devices.

They evaluate on UINavBench, iOSWorld and AndroidWorld. The online RL agents, they report, substantially outperform both SFT and terminal-only RL baselines, and training converges faster. The headline results concern the 8B RL agents: they achieve nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments. On real iOS applications they outperform Gemini-3.5-Flash and tie with Gemini-3.1-Pro.

Key facts

  • The framework trains UI navigation agents with online RL directly on live iOS apps, splitting the agent into a Planning agent and a UI Grounding model.
  • An LLM-as-a-Judge supplies dense step-level rewards plus a terminal task-completion reward, and the agent is optimized with PPO using step-level advantage estimation.
  • GiGPPO, a neighbor-conditioned value estimation method, aims to stabilize the critic using reward signals from visually similar states across concurrent trajectories.
  • The infrastructure orchestrates hundreds of concurrent virtual iOS devices.
  • The authors report that 8B RL agents outperform Gemini-3.5-Flash and tie with Gemini-3.1-Pro on real iOS apps, and reach nearly a sevenfold gain over SFT in out-of-distribution generalization to Android.

Why it matters

The paper targets a cost problem the authors name directly: frontier models handle UI navigation well, but their inference costs prohibit everyday use. A small 8B agent that ties Gemini-3.1-Pro on real iOS applications, if the result holds up, would point to a cheaper route. The authors also argue that SFT and offline RL models memorize static trajectories and cannot recover from errors, which is what training online on live apps is meant to fix.

Who it affects

Researchers building UI and mobile agents are the direct audience, especially those working with SFT or offline RL pipelines on fixed trajectories. Teams that want to run navigation agents on iOS and cannot afford frontier-model inference are the intended beneficiaries of the approach.

How to use it

The text describes a method, not a product. The recipe is: split the agent into a Planning agent and a UI Grounding model, train online on live iOS apps, score with an LLM-as-a-Judge that gives step-level and terminal rewards, optimize with PPO using step-level advantages, and use GiGPPO to steady the critic. The setup relies on infrastructure running hundreds of concurrent virtual iOS devices. The text does not mention released code, models or data.

How solid is it

This is a preprint listed on Hugging Face Papers, and the text available is the abstract, so all results are the authors' own claims. Evaluation covers UINavBench, iOSWorld and AndroidWorld. The text gives no absolute success rates, so the size of the gains in absolute terms is unknown. The sevenfold figure is relative to the SFT baseline and concerns out-of-distribution generalization to Android, not iOS performance. Authors and institutions are not named in the text.

Risks and caveats

The nearly sevenfold gain is relative; without the SFT baseline's absolute Android score it is hard to judge how large the gain is. The text does not say which base model the 8B agents use, and it gives no training cost, training time or inference cost figures, even though inference cost is the stated motivation. Rewards depend on an LLM-as-a-Judge, so the training signal is only as good as that judge. The comparison with Gemini-3.5-Flash and Gemini-3.1-Pro is reported on real iOS applications; the tie with Gemini-3.1-Pro is a tie, not a win.