Newest papers

What Novice Operators See Changes What the Policy Learns: A User Study of Keypoint Hand Visual Representation for Teleoperated Demonstration

Amane Yamaguchi, Shohei Kamiguchi, Masashi Hamaya, Shigeo Yoshida, Takuji Narumi, Mai Nishimura · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.27907 · cited by 0 works (OpenAlex) · open-access PDF

Korean

시뮬레이션 포인트 학습은 점점 더 초보자 운영자에 의존합니다. 수와 실험 간 일관성에 전문가의 데모가 부족합니다. 운영자는 그들이 보여주는 것에 따라 행동하지만 로봇의 렌더링이 그들의 시뮬레이션에 영향을 미치는 방법, 그리고 그들에 대해 훈련 된 정책은 검토되지 않았습니다. 우리는 PointHand로이 질문을 연구합니다.이 시뮬레이션은 여러 손가락 로봇의 손에 성공적인 키 포인트가 손가락과 관절에 단단한 키 포인트로 강조되는 텔레오페라 인터페이스입니다. 동일한 키 포인트는 정책을 관찰 할 수 없습니다. 시뮬레이션 된 mug-in-box 작업에서 18 명의 초보자와 4 명의 전문가가 3 개의 렌더링

Machine translation (Meta M2M100); the English original follows.

Abstract

Imitation learning increasingly relies on novice operators, whose demonstrations fall short of experts' in number and inter-trial consistency. Operators act on what they are shown, yet how the rendering of the robot affects their demonstrations, and the policies trained on them, has not been examined. We study this question with PointHand, a teleoperation interface that renders a multi-fingered robot hand as sparse keypoints at its fingertips and joints, the same keypoints a policy can observe. In a simulated mug-in-box task, 18 novices and 4 experts collected 2,200 trials under three renderings: the real shape of the hand, all keypoints emphasized, or the thumb and index finger alone emphasized. The novices shown the thumb and index finger emphasized collected 441 successful demonstrations from 600 trials, against 320 for those shown the real shape. We then trained diffusion policies on these demonstrations, replacing the scene point cloud and proprioceptive state they observe with the hand keypoints and two object keypoints. Within 400 steps these demonstrations went from the weakest of the three novice conditions under the point-cloud policy (46.3%) to the strongest under the keypoint policy (57.3%), a gain of +11.0 percentage points in all five seeds. Those collected with all keypoints emphasized changed by at most +0.7 percentage points, and in no more than three seeds. The value of a demonstration therefore depends on the rendering shown during collection and on what the policy observes together. A shared representation can pay off on both sides, in the number of successful demonstrations and in the success rate of the policy trained on them. Project page: https://omron-sinicx.github.io/point-hand/

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Teleoperation, Computer science, Rendering (computer graphics), Artificial intelligence, Robot, Computer vision

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange