Newest papers

DewHand: Direct Metric World-Space Hand Motion Reconstruction from Egocentric Videos

Naru Suzuki, Satoshi Ikehata, Takehiko Ohkawa, Yoichi Sato, Shuhei Kurita · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29936 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

Recuperar dispositivos de câmera-espaço-móvel métrico da câmera-clipper do vídeo egocêntrico suporta a compreensão das interações humanas e transferindo-as para agentes encarnados. Três desafios complicam esta tarefa: tubos de avaliação multi-estágio propagam erros de movimentos de mão e câmera estimados de forma independente; observações monoculares deixam a escala física ambígua; e a occlusão e o movimento de mão fora da visão em um sistema de coordenação do mundo compartilhado admitem múltiplas trajetórias de movimento plausíveis. Nós apresentamos o DewHand, um modelo unificado para a reconstrução conjunta direta da mão-espaço-móvel do mundo e do movimento da câmera HOT$. Para evitar erros de composição

Machine translation (Meta M2M100); the English original follows.

Abstract

Recovering metric world-space hand motion from egocentric video supports understanding human interactions and transferring them to embodied agents. Three challenges complicate this task: multi-stage pipelines propagate errors from independently estimated hand and camera motion; monocular observations leave physical scale ambiguous; and occlusion and out-of-view hands admit multiple plausible motion trajectories. We present DewHand, a unified model for direct joint reconstruction of world-space hand and camera motion. To avoid compounding errors from multi-stage estimation, DewHand jointly predicts hand and camera motion in a shared world coordinate system within a single-stage pipeline. To resolve metric-scale ambiguity, it combines scene-level geometric features from VGGT-$Ω$ with metric-space supervision, while hand-specific features preserve fine-grained articulation. To handle occlusion and out-of-view hands, we formulate reconstruction as conditional motion generation with flow matching. Under the same architecture, this generative formulation outperforms direct regression in both accuracy and temporal smoothness with only two sampling steps. Experiments on HOT3D and ARCTIC demonstrate improved world-space accuracy and temporal consistency over state-of-the-art methods. DewHand reduces rigid-aligned world-space error relative to StableHand from 32.9 to 28.4 mm on HOT3D and from 76.2 to 36.2 mm on ARCTIC. It also recovers camera scale within 10% on two thirds of HOT3D clips, versus about a quarter for SLAM. Moreover, in zero-shot evaluations on HoloAssist, H2O, and TACO, captured with different devices, DewHand achieves the lowest rigid-aligned world-space error (WA-MPJPE) on each dataset.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Artificial intelligence, Computer vision, Computer science, Motion (physics), Metric (unit), Motion capture

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange