Newest papers

GeoMem-VLA: Internalizing 3D Scene Understanding into Temporal Memory for Vision-Language-Action Models

Qinfan Zhang, Yifan Luo, Haoyuan Shi, Guanlin Liu, Han Dong, Haozhe Shan, Yong Dai, Xiangcong Ren, Xiaozhu Ju · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29866 · cited by 0 works (OpenAlex) · open-access PDF

French

La structure géométrique de la scène basse est essentielle pour la manipulation robotique et la compréhension du monde 3D. La plupart des modèles VLA reposent sur des représentations 2D des observations actuelles, tandis que ces approches conscientes de la géométrie nécessitent souvent des informations 3D supplémentaires. Cependant, la géométrie déduite indépendamment des observations individuelles est souvent rejetée plutôt que accumulée dans une représentation de tâche persistante. Notre point de vue clé sur la génération d'action est que la génération d'action a besoin d'une abstraction géométrique compacte et pertinente pour l'action, sans aligner directement les intégrations visuelles natives du VLM avec l'espace de mémoire actuel. Pendant la formation, un modèle de

Machine translation (Meta M2M100); the English original follows.

Abstract

Geometric structure is essential for robotic manipulation and understanding the 3D world. Most VLA models rely on 2D representations of current observations, while geometry-aware approaches often require additional 3D information. However, geometry inferred independently from individual observations is often discarded rather than accumulated into a persistent task representation. Our key insight is that action generation needs a compact, task-relevant geometric abstraction that can be continuously updated, rather than a complete, renderable 3D reconstruction at every step. We introduce GeoMem-VLA, a VLA framework that internalizes explicit 3D supervision into dedicated geometric tokens and maintains them as temporal memory for action generation, without directly aligning the VLM's native visual embeddings with the teacher feature space. During training, a frozen geometry foundation model supervises learnable GeoPlan tokens through a geometric flow-matching objective over the current scene point map and future end-effector tracks. These tokens are integrated with the memory of observation representations, where historical information is retrieved, adaptively fused with current observations, and consolidated to retain event frames as the task progresses. At inference, the policy retrieves historical representations and fuses them with the current working representation to predict actions, without invoking the 3D teacher or explicitly reconstructing the scene. After benchmark-specific downstream adaptation, GeoMem-VLA achieves 98.9% average success on LIBERO and 91.10% / 91.24% on RoboTwin's clean/randomized settings. Across five real-world tasks, GeoMem-VLA achieves 78.0% average success. These results suggest that internalizing and remembering 3D scene understanding is a promising approach to robot control.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Task (project management), Representation (politics), Feature (linguistics), Artificial intelligence, Abstraction

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange