Newest papers

Exploring Temporal Context-Aware Physical Grounding in Latent World Models

Yilei Man, Yang Chen, Jingcai Guo · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28483 · cited by 0 works (OpenAlex) · open-access PDF

Russian

Учение латентной динамики является фундаментальным для построения моделей мира, обусловленных действием для планирования. Среди существующих семей, Joint Embedding Predictive Architectures (JEPAs) предлагают естественную структуру путем моделирования переходов непосредственно в латентном пространстве, а не на уровне пикселей. Однако, две проблемы остаются в моделях мира, основанных на JEPA: (i) физическое заземление не имеет явной агрегации временного контекста и (ii) планирование действий всегда чрезмерно зависит от распределения общих образцов. Для этого мы предлагаем $\mathcal{G}-JEPA, который включает временный контекст в физическое заземление для моделей света, основанных на JEPA, и пары изученных моделей с предварительно ориентированным планирова

Machine translation (Meta M2M100); the English original follows.

Abstract

Learning latent dynamics is fundamental to building action-conditioned world models for planning. Among existing families, Joint Embedding Predictive Architectures (JEPAs) offer a natural framework by modeling transitions directly in latent space rather than at the pixel level. However, two issues remain in JEPA-based world models: (i) physical grounding lacks explicit aggregation of temporal context and (ii) action planning always overly relies on generic sampling distributions. To this end, we propose $\mathcal{G}$-JEPA, which incorporates temporal context into physical grounding for latent world models and couples the learned models with prior-guided action planning. Specifically, during learning, we incorporate a shared causal temporal module that aggregates temporal context within observation and prediction sequences under physical state supervision, grounding both in task-relevant physical states. When planning, we further introduce action inference through stepwise conditioning of a generative action distribution on the current and goal latent representations together with their evolving discrepancy, yielding anchored proposals that guide model-based online refinement. Together, physical state supervision guides temporal representation learning, while the action prior guides search through the learned world model. Experiments demonstrate improved planning performance on manipulation tasks, including tasks with increasing temporal separation between initial and goal observations. Code is publicly available at https://github.com/myl11045/G-JEPA.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Action (physics), Inference, Context (archaeology), Representation (politics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange