Newest papers

Physical Canvas: A Vision Tactile World Model For Robot Policy Evaluation And Learning

Zebin Xing, Yuhang Zheng, Yupeng Zheng, Yichen Zhang, Zichen Li, Shangqing Zhou, Yujie Zang, Yuchen Liu, Zhongpu Xia, Haoran Li, Qichao Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28613 · cited by 0 works (OpenAlex) · open-access PDF

Arabic

تقييم الأوضاع والسياسات الروبوتية بعد التدريب من خلال التفاعل الفكري منخفضة التكلفة ومثيرة للاهتمام، مما يجعله نموذجاً عالمياً منخفضاً عن التدابير مثل المحاكاة التفاعلية. ومع ذلك، فإن التوقعات المنطقية من الروبوت يمكن أن تشكل دموية الاتصال، وتوفير ردود فعل غير موثوقة لتقييم السياسات والتعلم. نحن نقدم PhysCanvas، نموذجاً عالمياً منخفضاً عن التفاعل الفكري من خلال التفاعل الفكري منخفضاً عن التفاعل الفكري من خلال التفاعل الفكري من خلال التفاعل الفكري من خلال التفاعل الفكري من خلال التفاعل الفكري. PhysCanvas يوفر

Machine translation (Meta M2M100); the English original follows.

Abstract

Evaluating and post-training robot policies through physical interaction is costly and difficult to scale, motivating action-conditioned world models as interactive simulators. However, visually plausible predictions can misrepresent contact dynamics, providing unreliable feedback for policy evaluation and learning. We introduce PhysCanvas, a state-consistent visuo-tactile world model for contact-rich robot manipulation. PhysCanvas unifies multi-view RGB, tactile images, marker displacement, task state, and actions within a unified vision space. All modalities share a video VAE and pretrained generative backbone, allowing optical touch to reuse visual and temporal priors. Motion images derived through forward kinematics and camera projection provide spatially aligned action conditioning, while an initial RGB anchor and updated reference observations guide sequential generation. Our consistency-aware post-training uses the frozen Scene Reward Model, Modality Reward Model, and State Reward Model to supervise scene preservation, visual-tactile compatibility, and agreement with real task outcomes. The resulting world model supports closed-loop policy rollouts for evaluation and provides imagined interactions and rewards for GRPO policy updates. Across four real-world manipulation tasks, when policies are rolled out in the world model, State Reward Model supervision increases the arithmetic mean of task-wise Pearson correlations between world-model and real-world success rates from 0.602 to 0.885. Policy post-training in the reward-guided world model further increases $π_{0.5}$'s mean real-world success rate from a 59.1% SFT baseline to 70.8%. These results demonstrate the value of unified multimodal modeling and consistency-aware post-training for robot policy evaluation and learning. Project page: https://zebinx.github.io/PhysCanvas_HomePage/.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Modality (human–computer interaction), Task (project management), Robot, Robotics

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange