Newest papers

Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent Training

Zezhou Wang, Hanzhang Zhou, Panrong Tong, Quyu Kong, Pengxiang Li, Steven C. H. Hoi, Zhuzhong Qian, Yue Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30792 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

GUIの強化学習をスケーリングするには、実際のアプリの行動を維持する環境が必要です。しかし、コードと行動の結論を組み合わせることは、間違った状態の移行を伴う実行可能な騒動に部分的な痕跡を変えることができます。我々は、グローバルな学習者適応タスクと実行に基づく環境の増大を分離する「textbf{Trace2World」を導入します。それは、共有された、ソース関連のアプリモデルに痕跡を調和し、その後実行可能な環境を構築します。ソースワークフローの独立した再生は、制限された実装修理のための不一致をローカリズム化します。検証された世界は、ソースがサポートする行動の中で、グローバルな

Machine translation (Meta M2M100); the English original follows.

Abstract

Scaling GUI reinforcement learning requires environments that preserve real-app behavior. Yet coupling behavior inference with coding can turn partial traces into runnable mocks with incorrect state transitions. We introduce \textbf{Trace2World}, which separates global behavior reconstruction from implementation. It reconciles traces into a shared, source-linked app model, then compiles an executable environment. Independent replay of source workflows localizes mismatches for bounded implementation repair. Validated worlds then support learner-adaptive task and environment augmentation within source-supported behavior. From 1,570 traces, Trace2World constructs 50 environments. On 200 workflows across ten apps, 91.1% are reproduced end to end, versus 59.8% for the strongest matched task-local baseline. Source-grounded validation retains 94.0% of simulated training gains on real devices, versus 54.7% under self-validation. Learner guidance raises benchmark success from 56.9% to 61.1%. Training in Trace2World further improves the model by 20.2 points on our benchmark, 5.4--16.2 points across four external GUI benchmarks, and 16.3 points on 100 real-device tasks. Reliable reconstruction separates what behavior source evidence supports from whether the implementation preserves it.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Scalability, Executable, Workflow, Bounded function, Task (project management)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange