Ruicheng Li, Qixiu Li, Yu Deng, Sicheng Xu, Jiaolong Yang, Baining Guo · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.26780 · cited by 0 works (OpenAlex) · open-access PDF
La récupération des trajectoires manuelles métriques 3D à partir de telles vidéos nécessite l'estimation du mouvement de la main et de l'ego-motion inférieure de la caméra pour exprimer les mains dans un cadre de référence cohérent. De nombreux pipelines existants récupèrent le mouvement de la main et de la caméra dans des étapes distinctes, combinant la détection de la main, la reconstruction des pipelines, le suivi de la caméra et le raffinement de la trajectoire. Nous présentons V-JEPA Hand, un modèle vidéo simple à l'avant-garde qui reconstruit conjointement le mouvement bilatéral de la main et le mouvement de la caméra à partir de clips egocentriques. Il utilise un enc
Machine translation (Meta M2M100); the English original follows.
Egocentric video supports diverse applications, including robot learning, augmented reality, and human activity understanding. Recovering metric 3D hand trajectories from such video requires estimating both hand motion and camera egomotion to express the hands in a consistent reference frame. Many existing pipelines recover hand and camera motion in separate stages, combining hand detection, hand reconstruction, camera tracking, and trajectory refinement. We present V-JEPA Hand, a simple feed-forward video model that jointly reconstructs bilateral hand motion and camera motion from egocentric clips. It uses a self-supervised V-JEPA 2.1 encoder to extract spatio-temporal features from full-frame video and a query-based predictor to decode these features into per-frame hand poses, hand presence, and camera motion. We use the predicted camera motion to express the hand trajectories in a common world frame. The model accepts known focal lengths or predicts them when unavailable. It reconstructs each clip in a single forward pass. We train both components end to end in a single stage on a mixture of diverse pseudo-labelled videos and high-precision 3D annotations. A single model achieves the lowest coverage-aware mean per-joint position error (MPJPE) among comparable baselines on ARCTIC, HOT3D, and EgoStandard. It also yields lower world-frame trajectory errors than the compared SLAM-based pipelines. Ablations support the benefits of video pretraining, temporal context, and mixed supervision. V-JEPA Hand achieves a forward-pass throughput of 93.4 FPS on a single RTX A6000. Project page: https://qft-333.github.io/V-JEPA-Hand-Page/.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer vision, Artificial intelligence, Computer science, Trajectory, Motion (physics), Camera auto-calibration
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26780, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange