Newest papers

PiHand: Latent Hand-Object Interaction Priors for Physics-Based Policy Learning

Niklas Berndt, Xianghui Xie, Enes Duran, Yixiao Wang, Gerard Pons‐Moll · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31299 · cited by 0 works (OpenAlex) · open-access PDF

German

Die Synthese physikalisch zugrunde liegender Datenkompetente Hand-Objekt-Interaktionen ist wichtig für VR/AR, Gaming und Robotik, bleibt jedoch herausfordernd, weil subtile Kontaktdynamik, vielfältige Objekt-Geometrien und Ziel-Trajectories gemeinsam respektiert werden müssen, während natürliche Griffs aufrechterhalten werden. Bestehende Methoden verfolgen entweder Mocap-Interaktion-bewußte Hand-Bewegungssynthese-Rahmen mit begrenzter Übertragung auf neue Objekte, oder verlassen sich auf Hand-only generative Priors, die in der Simulation gesammelte Hand-Objekt-Bewegung nicht erfassen, wobei Kontaktfriedlichkeit und Mustereffizienz geopfert werden müssen. Um diese Herausforderungen anzugehen

Machine translation (Meta M2M100); the English original follows.

Abstract

Synthesizing physically grounded dexterous hand-object interactions is important for VR/AR, gaming, and robotics, yet remains challenging because subtle contact dynamics, diverse object geometries, and target trajectories must be jointly respected while maintaining natural grasps. Existing methods either track mocap trajectories with limited transfer to novel objects, or rely on hand-only generative priors that fail to capture coupled hand-object motion, sacrificing contact fidelity and sample efficiency. To tackle these challenges, we introduce PiHand, a Physically grounded Interaction-aware Hand motion synthesis framework. PiHand first trains expert policies that track mocap demonstrations in simulation, distills them into a unified hand-object interaction prior using a goal-conditioned VAE, and finally solve diverse dexterous manipulation tasks through bounded residual exploration in its latent space. With only a single reward and no complex curriculum training, this yields general tabletop manipulation policies that generalize to unseen objects and trajectories on GRAB and DexYCB. The same prior transfers to object catching and bimanual handover, two tasks that are rarely seen in its training data. Across all tasks, PiHand consistently outperforms baselines in all metrics. We will release the processed data and model checkpoints.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Object (grammar), Prior probability, Machine learning, Motion (physics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange