Newest papers

DewHand: Direct Metric World-Space Hand Motion Reconstruction from Egocentric Videos

Naru Suzuki, Satoshi Ikehata, Takehiko Ohkawa, Yoichi Sato, Shuhei Kurita · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29936 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

從自我中心視頻中恢復攝像機的測量世界空間手動裝置支持人類相互作用的理解,並將其轉移到體現的代理人。三個挑戰複雜了這個任務:多階段管道從獨立估計的手動和攝像機運動中傳播錯誤;單眼觀測讓物理尺度不明確;並在共享世界坐標系統上預測人類互動和外觀手動的理解。 我們介紹了DewHand,一個為世界空間手和攝像機運動的直接聯合重建的統一模型。 為了避免多階段估計的複合錯誤,DewHand共同預測手動和外觀手動在單一階段時間錯誤系統內的共享世界坐標。 為了確保低階段誤差的解決,

Machine translation (Meta M2M100); the English original follows.

Abstract

Recovering metric world-space hand motion from egocentric video supports understanding human interactions and transferring them to embodied agents. Three challenges complicate this task: multi-stage pipelines propagate errors from independently estimated hand and camera motion; monocular observations leave physical scale ambiguous; and occlusion and out-of-view hands admit multiple plausible motion trajectories. We present DewHand, a unified model for direct joint reconstruction of world-space hand and camera motion. To avoid compounding errors from multi-stage estimation, DewHand jointly predicts hand and camera motion in a shared world coordinate system within a single-stage pipeline. To resolve metric-scale ambiguity, it combines scene-level geometric features from VGGT-$Ω$ with metric-space supervision, while hand-specific features preserve fine-grained articulation. To handle occlusion and out-of-view hands, we formulate reconstruction as conditional motion generation with flow matching. Under the same architecture, this generative formulation outperforms direct regression in both accuracy and temporal smoothness with only two sampling steps. Experiments on HOT3D and ARCTIC demonstrate improved world-space accuracy and temporal consistency over state-of-the-art methods. DewHand reduces rigid-aligned world-space error relative to StableHand from 32.9 to 28.4 mm on HOT3D and from 76.2 to 36.2 mm on ARCTIC. It also recovers camera scale within 10% on two thirds of HOT3D clips, versus about a quarter for SLAM. Moreover, in zero-shot evaluations on HoloAssist, H2O, and TACO, captured with different devices, DewHand achieves the lowest rigid-aligned world-space error (WA-MPJPE) on each dataset.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Artificial intelligence, Computer vision, Computer science, Motion (physics), Metric (unit), Motion capture

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange