Newest papers

PiHand: Latent Hand-Object Interaction Priors for Physics-Based Policy Learning

Niklas Berndt, Xianghui Xie, Enes Duran, Yixiao Wang, Gerard Pons‐Moll · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31299 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

Síntese de dados fisicamente aterrados de interações de mão-objeto hábeis é importante para VR/AR, jogos e robótica, mas permanece desafiador porque a dinâmica de contato sutil, geometria de objeto diversificada e trajetórias de alvo devem ser respeitadas conjuntamente, enquanto mantêm os grampos naturais. Métodos existentes ou rastrear as trajetórias de mão-objeto com transferência limitada para novos objetos, ou depender de pré-generações de mão-só que não capturam o movimento de mão-objeto acoplado, sacrificando a fidelidade de contato e a eficiência da amostra. Para enfrentar esses desafios, introduzimos PiHand, um mecanismo de síntese de movimento de mão fisicamente aterrado de interação

Machine translation (Meta M2M100); the English original follows.

Abstract

Synthesizing physically grounded dexterous hand-object interactions is important for VR/AR, gaming, and robotics, yet remains challenging because subtle contact dynamics, diverse object geometries, and target trajectories must be jointly respected while maintaining natural grasps. Existing methods either track mocap trajectories with limited transfer to novel objects, or rely on hand-only generative priors that fail to capture coupled hand-object motion, sacrificing contact fidelity and sample efficiency. To tackle these challenges, we introduce PiHand, a Physically grounded Interaction-aware Hand motion synthesis framework. PiHand first trains expert policies that track mocap demonstrations in simulation, distills them into a unified hand-object interaction prior using a goal-conditioned VAE, and finally solve diverse dexterous manipulation tasks through bounded residual exploration in its latent space. With only a single reward and no complex curriculum training, this yields general tabletop manipulation policies that generalize to unseen objects and trajectories on GRAB and DexYCB. The same prior transfers to object catching and bimanual handover, two tasks that are rarely seen in its training data. Across all tasks, PiHand consistently outperforms baselines in all metrics. We will release the processed data and model checkpoints.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Object (grammar), Prior probability, Machine learning, Motion (physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange