Newest papers

GPT-6-Astra Understands Touch and Outperform Vision--Tactile--Language--Action Models

Ziyong Wang, Yue Lin, Runxuan Huang, Liang Lin, Dongyu Zhang, Guangrun Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28457 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

一般用途のマルチモダルモデルは、直接的なロボットアクションポリシーとして約束を示したが、実際の世界の相互作用の基礎となる物理学を把握し、行動を指導するためのその理解を適用できるかどうかは未だに未だに研究されていない。われわれは、代表的なテストベッドとしてコンタクト豊富な操作を使用して、直接的なロボットアクションポリシーの具体的な例としてタクティル理解を検討し、これらの質問を調査している。コンタクト豊富な操作はタクティルフィードバックを解釈し、障害の後のアクションを調整することを必要とする。既存の学習されたタクティルポリシーは、専門的なプレトレーニングおよびタスク毎の最適化を通じてこれらの能力を調べている。われ

Machine translation (Meta M2M100); the English original follows.

Abstract

General-purpose multimodal models have shown promise as direct robot action policies, yet whether they grasp the physics underlying real-world interactions and can apply that understanding to guide action remains underexplored. We investigate this question using contact-rich manipulation as a representative testbed, examining tactile understanding as a concrete instance of this broader capability. Contact-rich manipulation requires interpreting tactile feedback and adjusting actions after obstruction. Existing learned tactile policies acquire these capabilities through specialized pretraining and per-task optimization, with substantial data and training costs. We present GPT-as-Tactile-Policy, which summarizes expert demonstrations into task guidance and tactile references for contact assessment and corrective action. GPT-6-Astra combines this context with multimodal observations and interaction history to generate control commands and adjust actions through contact feedback. Without additional robotics-specific pretraining or per-task policy fine-tuning, our method achieves 93.8\% mean success across five UniVTAC tasks. Overall performance is comparable to state-of-the-art policies, with higher success on selected tasks. On Insert Hole, tactile feedback improves success from 31\% to 99\%. Execution analysis further demonstrates contact-guided probing and correction, supporting manipulation progress through explicit expert experience and online feedback.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Task (project management), GRASP, Action (physics), Computer science, Human–computer interaction, Context (archaeology)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange