Newest papers

PiHand: Latent Hand-Object Interaction Priors for Physics-Based Policy Learning

Niklas Berndt, Xianghui Xie, Enes Duran, Yixiao Wang, Gerard Pons‐Moll · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31299 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

对VR/AR、游戏和机器人来说,控制物理地化数据巧妙的手对象交互是很重要的,但仍然具有挑战性,因为微妙的接触动态、多元的对象形态和目标轨道必须共同得到尊重,同时保持自然的把握。现有的方法要么跟踪Mocap交互意识手动合成框架,以有限的转移到新物体,要么依赖于仅在模拟中跟踪手对象运动的手对象生成前者,无法捕捉相同的手对象运动,牺牲了接触忠诚度和样本效率。为了应对这些挑战,我们引入PiHand,一个物理地化互动意识手动合成框架。PiHand首先训练专家政策,在模拟中跟踪Mocap任务示范,将它们分配成

Machine translation (Meta M2M100); the English original follows.

Abstract

Synthesizing physically grounded dexterous hand-object interactions is important for VR/AR, gaming, and robotics, yet remains challenging because subtle contact dynamics, diverse object geometries, and target trajectories must be jointly respected while maintaining natural grasps. Existing methods either track mocap trajectories with limited transfer to novel objects, or rely on hand-only generative priors that fail to capture coupled hand-object motion, sacrificing contact fidelity and sample efficiency. To tackle these challenges, we introduce PiHand, a Physically grounded Interaction-aware Hand motion synthesis framework. PiHand first trains expert policies that track mocap demonstrations in simulation, distills them into a unified hand-object interaction prior using a goal-conditioned VAE, and finally solve diverse dexterous manipulation tasks through bounded residual exploration in its latent space. With only a single reward and no complex curriculum training, this yields general tabletop manipulation policies that generalize to unseen objects and trajectories on GRAB and DexYCB. The same prior transfers to object catching and bimanual handover, two tasks that are rarely seen in its training data. Across all tasks, PiHand consistently outperforms baselines in all metrics. We will release the processed data and model checkpoints.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Object (grammar), Prior probability, Machine learning, Motion (physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange