Newest papers

UniWALM: Unified World Action Language Model for Embodied Robot Control

Rahul Sajnani, Diego Romeres, Mingyuan Zhang, Kai Gao, Carl Qi, Baichuan Huang, Srinath Sridhar, Dylan Randle, Lillian J. Ratliff, Jose Augusto Lima Barreiros, Manikantan Nambi, Jeroen Vanbaar · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28626 · cited by 0 works (OpenAlex) · open-access PDF

Arabic

ونحن نقدم UniWALM، نموذجًا متكاملًا للغة العملية واللغة التي تعلم الفيديو والعمل واللغة في ملاحظة مشتركة واحدة، حيث كل نموذج هو أيضاً إشارة التخطيط وهدف التحكم، ونحن نسأل: هل يمكن إنتاج اللغة المشتركة معًا إثراء هذا التخطيط المشترك، وتحسين التمثيل وتدعم السلوك الذكي المعتمد على الذاكرة؟ على عكس الأساليب السابقة التي تنص على نموذج التخطيط أو الذاكرة التخطيطية على نفس نموذج التخطيط والتخطيط وتشغيلها كخطوة منفصلة، UniWALM يتكامل كل ثلاثة نموذجات على نطاق واحد من نموذج التخطيط المشتر

Machine translation (Meta M2M100); the English original follows.

Abstract

Motivated by the human ability to integrate experience across language, perception, and action, we introduce UniWALM, a unified world-action-language model that learns video, action, and language in a single shared representation, where each modality is both a conditioning signal and a supervision target. We ask: can jointly generating language enrich this shared representation, improving manipulation and supporting memory-dependent behavior? Unlike prior approaches that attach a reasoning or vision-language model to a video-action backbone and run them as distinct stages, UniWALM unifies all three modalities inside a single mixture-of-transformers backbone whose per-modality diffusion heads co-denoise synchronously, decoding subtask and textual memory in the same pass as action with no separate planning stage; the generated language is re-consumed on the next step, closing a language-as-memory loop without an external planner or memory bank. To keep the action stream reliable under this richer objective, we further introduce Flow Matching with Correction (FMC), a training objective that teaches recovery from perturbed action trajectories at no inference-time cost. UniWALM consistently improves over same-backbone baselines (e.g., FastWAM) on compositional pick-and-place (MolmoSpaces) and across the memory-dependent subset of RMBench, where it reaches 34.5% without any pretrained language or reasoning model and exceeds Mem-0's keyframe-equipped setup (28.5%) using only a single anchor frame. On a real-world Find-the-Item task, generating textual memory lifts success from 11.1% to 61.1%, while state-of-the-art VLAs stay at or below chance (<=33.3%).

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Action (physics), Language model, Artificial intelligence, Robot, Matching (statistics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange