Newest papers

UniWALM: Unified World Action Language Model for Embodied Robot Control

Rahul Sajnani, Diego Romeres, Mingyuan Zhang, Kai Gao, Carl Qi, Baichuan Huang, Srinath Sridhar, Dylan Randle, Lillian J. Ratliff, Jose Augusto Lima Barreiros, Manikantan Nambi, Jeroen Vanbaar · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28626 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

我们引入了 UniWALM,一个统一的世界动作语言模型,该模型在单个共享记忆中学习视频,动作和语言,其中每个模式既是一个调节信号,又是一个监督目标。我们问:可以联合生成语言来丰富这种共享表示,改进操纵和支持记忆依赖的行为?与以前的方法不同,这些方法将推理或视觉语言模型连接到相同的视频动作背骨,并运行它们作为不同的阶段,UniWALM将所有三种模式统一在一起,在单一的变形背骨中,在一个单一的变形背骨上,在一个单一的组合中,在一个单一的变形背骨上,在一个单一的模型中,在一个单一的动作中进行调节,在一个单一的动作中

Machine translation (Meta M2M100); the English original follows.

Abstract

Motivated by the human ability to integrate experience across language, perception, and action, we introduce UniWALM, a unified world-action-language model that learns video, action, and language in a single shared representation, where each modality is both a conditioning signal and a supervision target. We ask: can jointly generating language enrich this shared representation, improving manipulation and supporting memory-dependent behavior? Unlike prior approaches that attach a reasoning or vision-language model to a video-action backbone and run them as distinct stages, UniWALM unifies all three modalities inside a single mixture-of-transformers backbone whose per-modality diffusion heads co-denoise synchronously, decoding subtask and textual memory in the same pass as action with no separate planning stage; the generated language is re-consumed on the next step, closing a language-as-memory loop without an external planner or memory bank. To keep the action stream reliable under this richer objective, we further introduce Flow Matching with Correction (FMC), a training objective that teaches recovery from perturbed action trajectories at no inference-time cost. UniWALM consistently improves over same-backbone baselines (e.g., FastWAM) on compositional pick-and-place (MolmoSpaces) and across the memory-dependent subset of RMBench, where it reaches 34.5% without any pretrained language or reasoning model and exceeds Mem-0's keyframe-equipped setup (28.5%) using only a single anchor frame. On a real-world Find-the-Item task, generating textual memory lifts success from 11.1% to 61.1%, while state-of-the-art VLAs stay at or below chance (<=33.3%).

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Action (physics), Language model, Artificial intelligence, Robot, Matching (statistics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange