Newest papers

Agent World Model in the Wild: Computer-Use Agent Environment Synthesis

Zhaoyang Wang, Qianhui Wu, Haorui Wang, Simon Yu, Baolin Peng, Canwen Xu, Hao Cheng, Wenlin Yao, Boyi Liu, Yite Wang, Jianfeng Gao, Huaxiu Yao · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26828 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

然而,在现实世界的计算机系统中直接扩展这种学习是具有挑战性的,并且可以引入大量的公共安全风险。因此,我们提出了从人类数据的学习到通过与世界的互动进行学习的可扩展管道(Agent World Model in the Wild,WildAWM),用于合成可控环境和培训通用计算机使用代理(CUA)的任务。虽然现有的方法通常专注于单个代理交互界面,我们构建了可执行的环境,这些环境可以联合支持图形用户界面,模型背景协议服务和命令线接口,允许代理人灵活地结合视觉交互,工具使用和代码执行环境。WildAWM完全由开放源代码交互模型提供动力,这表明高质量的计算机交互代码环境仍然可以生成,而没有拥有

Machine translation (Meta M2M100); the English original follows.

Abstract

Agent learning is shifting from learning from human data toward learning through interaction with the world. However, scaling such learning directly in real-world computer systems is challenging and can introduce substantial security risks. We therefore propose Agent World Model in the Wild (WildAWM), a scalable pipeline for synthesizing controllable environments and tasks for training general computer-use agents (CUAs). Whereas existing approaches typically specialize in a single agent interaction interface, we construct executable environments that jointly support graphical user interfaces, model context protocol services, and command-line interfaces, allowing agents to flexibly combine visual interaction, tool use, and code execution. WildAWM is entirely powered by open-source models, demonstrating that high-quality computer-use environments can be generated without proprietary frontier models. Beyond environments, we address three recurring challenges in task synthesis: ensuring solvability, maintaining sufficient difficulty, and producing reliable verifiers, by introducing execution-based quality gates that combine heterogeneous agent rollouts, cross-judge consistency checks, and counterfactual probes of judgment robustness. The resulting pipeline synthesizes 341 long-horizon environment-task pairs that can require over 100 agent steps to solve and remain challenging even for frontier models, with GPT-6-Astra solving only 60% of them. We train two Qwen models of different sizes exclusively in these synthetic worlds. Both models achieve substantial gains across 8 out-of-distribution benchmarks spanning web, claw, and computer-use agents, demonstrating that learning in WildAWM's synthesized worlds generalizes broadly to unseen computer-use tasks. The code is publicly available at https://github.com/MSR-Orchard/WildAWM.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Executable, Context (archaeology), Scalability, Pipeline (software), Artificial intelligence

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange