Newest papers

From Predictive Geometry to Action: Learning Complementary Predictive Representations for Embodied Manipulation

Xiaoxu Xu, Hao Li, Hongtao Zhang, Yili Tang, Jiafei Cao, Junhao Cai, Haoxiang Ma, Weixin Li, Weinan Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28164 · cited by 0 works (OpenAlex) · open-access PDF

Korean

유효한 LIBI 조작은 현재 장면 기하학에 대해뿐만 아니라 상호 작용 중에 어떻게 진화 할 것인지를 포착하기위한 정책을 필요로합니다. 기존 접근 방식은 현재 관측에서 지하학 정보를 사용하여 정책을 확대하거나 보조 감독으로 미래 예측을 도입하여 예측 기하학이 액션 생성을 위해 어떻게 표현되어야하는지 공개합니다. 따라서, 우리는 두 단계 프레임 워크를 제안합니다. 여러 뷰 비디오 클립에서 예측을 보완하는 예측을 배울 수있는 액션 감독 작업은 다중 뷰 비디오 클립에서 예측을 보완합니다. 사전 훈련 중에, 냉동 된 3D 기초 모델은 예측 기하학을 예측 기하학에 압축하고 최적화하여 장면 구조와 시간 진화를

Machine translation (Meta M2M100); the English original follows.

Abstract

Effective manipulation requires a policy to capture not only about the current scene geometry, but also about how that geometry will evolve under interaction. Existing approaches either augment policies with geometric information from current observations or introduce future prediction as auxiliary supervision, leaving open how predictive geometry should be represented for action generation. Therefore, we propose a two-stage framework, which learns complementary predictive representations from multi-view video clips. During pretraining, a frozen 3D foundation model provides dense geometric features that are compressed into predictive geometric representations and optimized to preserve scene structure and its temporal evolution. Action supervision is then applied to geometric transitions to extract a complementary motor representation that emphasizes action-relevant transitions. During downstream post-training, predictive geometry is transferred through geometry queries, while the motor representation guides intermediate features along the action-generation pathway. Future observations are used only as privileged supervision during training and are not required at inference time. Across LIBERO, LIBERO-Plus, SimplerEnv, and real-world bimanual and single-arm tasks, our approach consistently improves downstream policies, including an absolute gain of 10.4 percentage points on the WidowX evaluation. We further demonstrate transfer across multiple VLA architectures and WAMs, with improvements in both future-world and action prediction. The code will be available at https://github.com/xuxiaoxxxx/CPR.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Representation (politics), Artificial intelligence, Computer science, Action (physics), Inference, Embodied cognition

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange