Newest papers

GPT-6-Astra Understands Touch and Outperform Vision--Tactile--Language--Action Models

Ziyong Wang, Yue Lin, Runxuan Huang, Liang Lin, Dongyu Zhang, Guangrun Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28457 · cited by 0 works (OpenAlex) · open-access PDF

German

Multimodale Modelle mit allgemeinem Zweck haben das Versprechen als direkte Roboter-Aktionsrichtlinien gezeigt, ob sie die physikalische Grundlage der realen Interaktionen erfassen und dieses Verständnis anwenden können, um die Aktion zu leiten, bleibt unerforscht. Wir untersuchen diese Frage mit Hilfe von kontaktreichen Manipulationen als repräsentativem Testbett und untersuchen das taktile Verständnis als konkretes Beispiel dieser breiteren Fähigkeit. Kontaktreiche Manipulationen erfordern die Interpretation von taktilen Rückmeldungen und Anpassungsmaßnahmen nach Behinderung. Bestehende gelernte taktile Richtlinien erwerben diese Fähigkeiten durch spezialisierte Rückmeldungen und Pro-Task-Optimierung mit erheblichen Daten- und Trainingskosten. Wir präsentieren GPT-

Machine translation (Meta M2M100); the English original follows.

Abstract

General-purpose multimodal models have shown promise as direct robot action policies, yet whether they grasp the physics underlying real-world interactions and can apply that understanding to guide action remains underexplored. We investigate this question using contact-rich manipulation as a representative testbed, examining tactile understanding as a concrete instance of this broader capability. Contact-rich manipulation requires interpreting tactile feedback and adjusting actions after obstruction. Existing learned tactile policies acquire these capabilities through specialized pretraining and per-task optimization, with substantial data and training costs. We present GPT-as-Tactile-Policy, which summarizes expert demonstrations into task guidance and tactile references for contact assessment and corrective action. GPT-6-Astra combines this context with multimodal observations and interaction history to generate control commands and adjust actions through contact feedback. Without additional robotics-specific pretraining or per-task policy fine-tuning, our method achieves 93.8\% mean success across five UniVTAC tasks. Overall performance is comparable to state-of-the-art policies, with higher success on selected tasks. On Insert Hole, tactile feedback improves success from 31\% to 99\%. Execution analysis further demonstrates contact-guided probing and correction, supporting manipulation progress through explicit expert experience and online feedback.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Task (project management), GRASP, Action (physics), Computer science, Human–computer interaction, Context (archaeology)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange