RESEARCH · RESEARCH · #2925
Online RL with Planning‑Grounding Agent and GiGPPO improves UI navigation on live iOS apps
The paper introduces an online reinforcement learning framework that trains a decoupled Planning agent and UI Grounding model directly on live iOS apps, using an LLM-as-a-Judge to supply dense step-level and terminal rewards and optimizing with PPO plus a new neighbor-conditioned critic (GiGPPO). Evaluations on UINavBench, iOSWorld and AndroidWorld show the authors' 8B RL agents substantially outperform supervised-finetuned and terminal-only RL baselines, achieve ~7× better out-of-distribution generalization to Android, and narrowly beat or match large VLM baselines (outperforming Gemini-3.5-Flash and tying with Gemini-3.1-Pro) on real iOS tasks.
KEY POINTS
- The paper introduces an online reinforcement learning framework that trains a decoupled Planning agent and UI Grounding model directly on live iOS apps, using an LLM-as-a-Judge to supply dense step-level and terminal rewards and optimizing with PPO plus a new neighbor-conditioned critic (GiGPPO).
- Evaluations on UINavBench, iOSWorld and AndroidWorld show the authors' 8B RL agents substantially outperform supervised-finetuned and terminal-only RL baselines, achieve ~7× better out-of-distribution generalization to Android, and narrowly beat or match large VLM baselines (outperforming Gemini-3.5-Flash and tying with Gemini-3.1-Pro) on real iOS tasks.
- Training agents online on live apps with dense, LLM‑generated rewards and GiGPPO improves robustness and cross-platform generalization, addressing cost and brittleness limitations of large VLMs and offline-trained agents.
WHY IT MATTERS
Training agents online on live apps with dense, LLM‑generated rewards and GiGPPO improves robustness and cross-platform generalization, addressing cost and brittleness limitations of large VLMs and offline-trained agents.