Newest papers

Exploring Temporal Context-Aware Physical Grounding in Latent World Models

Yilei Man, Yang Chen, Jingcai Guo · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28483 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Di antara keluarga yang ada, Joint Embedding Predictive Architectures (JEPAs) menawarkan kerangka kerja alami dengan memodelkan transisi langsung di ruang latent daripada pada tingkat pixel. Namun, dua masalah tetap dalam model dunia berbasis JEPA: (i) grounding fisik tidak memiliki agregasi eksplisit dari konteks temporal dan (ii) planning tindakan selalu terlalu bergantung pada distribusi sampel umum. Untuk tujuan ini, kami mengusulkan $\mathcal{G}-JEPA, yang menggabungkan konteks temporal ke dalam grounding fisik untuk model dunia latent dan pasangan model yang dipelajari dengan perencanaan tindakan yang dipandu sebelumnya. Secara khusus, selama belajar, kami menggabungkan modul causal temporal bersama yang menggabungkan publik tujuan umum dalam pengamatan konteks temporal dan urutan perencanaan tugas di bawah pengawasan fisik, mendemonstrasikan keduanya kondisi fisik yang relevan

Machine translation (Meta M2M100); the English original follows.

Abstract

Learning latent dynamics is fundamental to building action-conditioned world models for planning. Among existing families, Joint Embedding Predictive Architectures (JEPAs) offer a natural framework by modeling transitions directly in latent space rather than at the pixel level. However, two issues remain in JEPA-based world models: (i) physical grounding lacks explicit aggregation of temporal context and (ii) action planning always overly relies on generic sampling distributions. To this end, we propose $\mathcal{G}$-JEPA, which incorporates temporal context into physical grounding for latent world models and couples the learned models with prior-guided action planning. Specifically, during learning, we incorporate a shared causal temporal module that aggregates temporal context within observation and prediction sequences under physical state supervision, grounding both in task-relevant physical states. When planning, we further introduce action inference through stepwise conditioning of a generative action distribution on the current and goal latent representations together with their evolving discrepancy, yielding anchored proposals that guide model-based online refinement. Together, physical state supervision guides temporal representation learning, while the action prior guides search through the learned world model. Experiments demonstrate improved planning performance on manipulation tasks, including tasks with increasing temporal separation between initial and goal observations. Code is publicly available at https://github.com/myl11045/G-JEPA.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Action (physics), Inference, Context (archaeology), Representation (politics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange