Newest papers

Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent Training

Zezhou Wang, Hanzhang Zhou, Panrong Tong, Quyu Kong, Pengxiang Li, Steven C. H. Hoi, Zhuzhong Qian, Yue Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30792 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Escalar el aprendizaje de amplificación de GUI requiere entornos que preserven el comportamiento de aplicaciones reales. No obstante, acoplar la inferencia del comportamiento con la codificación puede convertir rastros parciales en chistes ejecutables con transiciones de estado incorrectas. Introducimos \textbf{Trace2World}, que separa la reconstrucción del comportamiento global del aprendizaje y la ampliación del entorno dentro del comportamiento soportado por la fuente. De 1,570 rastros, Trace2World construye 50 entornos y luego compila un entorno ejecutable. En 200 flujos de trabajo a través de diez aplicaciones, la reproducción independiente de los flujos de trabajo de fuente localiza los malentendidos para la reparación de la implementación limitada. Los mundos validados luego apoyan la tarea adaptativa

Machine translation (Meta M2M100); the English original follows.

Abstract

Scaling GUI reinforcement learning requires environments that preserve real-app behavior. Yet coupling behavior inference with coding can turn partial traces into runnable mocks with incorrect state transitions. We introduce \textbf{Trace2World}, which separates global behavior reconstruction from implementation. It reconciles traces into a shared, source-linked app model, then compiles an executable environment. Independent replay of source workflows localizes mismatches for bounded implementation repair. Validated worlds then support learner-adaptive task and environment augmentation within source-supported behavior. From 1,570 traces, Trace2World constructs 50 environments. On 200 workflows across ten apps, 91.1% are reproduced end to end, versus 59.8% for the strongest matched task-local baseline. Source-grounded validation retains 94.0% of simulated training gains on real devices, versus 54.7% under self-validation. Learner guidance raises benchmark success from 56.9% to 61.1%. Training in Trace2World further improves the model by 20.2 points on our benchmark, 5.4--16.2 points across four external GUI benchmarks, and 16.3 points on 100 real-device tasks. Reliable reconstruction separates what behavior source evidence supports from whether the implementation preserves it.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Scalability, Executable, Workflow, Bounded function, Task (project management)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange