Newest papers

Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent Training

Zezhou Wang, Hanzhang Zhou, Panrong Tong, Quyu Kong, Pengxiang Li, Steven C. H. Hoi, Zhuzhong Qian, Yue Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30792 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Scaling GUI enhancement learning requires environments that preserve real-app behavior. Yet coupling behavior inference with coding can turn partial traces into runable mocks with incorrect state transitions. We introduce \textbf{Trace2World}, which separates global behavior reconstruction from implementation. It reconciles traces into a shared, source-linked app model, then compiles an executable environment. On 200 workflows across ten apps, 91.1% replay of source workflows are reproduced end to end, versus 59.8% for bounded implementation repair. Validated worlds then support learner-adaptive task and environment augmentation within source-supported behavior. From 1,570 traces, Trace2World builds 50 environments. On 200 workflows across ten apps, 9

Machine translation (Meta M2M100); the English original follows.

Abstract

Scaling GUI reinforcement learning requires environments that preserve real-app behavior. Yet coupling behavior inference with coding can turn partial traces into runnable mocks with incorrect state transitions. We introduce \textbf{Trace2World}, which separates global behavior reconstruction from implementation. It reconciles traces into a shared, source-linked app model, then compiles an executable environment. Independent replay of source workflows localizes mismatches for bounded implementation repair. Validated worlds then support learner-adaptive task and environment augmentation within source-supported behavior. From 1,570 traces, Trace2World constructs 50 environments. On 200 workflows across ten apps, 91.1% are reproduced end to end, versus 59.8% for the strongest matched task-local baseline. Source-grounded validation retains 94.0% of simulated training gains on real devices, versus 54.7% under self-validation. Learner guidance raises benchmark success from 56.9% to 61.1%. Training in Trace2World further improves the model by 20.2 points on our benchmark, 5.4--16.2 points across four external GUI benchmarks, and 16.3 points on 100 real-device tasks. Reliable reconstruction separates what behavior source evidence supports from whether the implementation preserves it.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Scalability, Executable, Workflow, Bounded function, Task (project management)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange