Newest papers

UniRAP: Towards Unified Part-level Physical Affordance Reasoning and Actionable Perception

Linfei Li, Ruining Hu, Lin Zhang, Zhong Wang, Fengyi Zhang, Ying Shen, Binqiang Wang, Xin Zhang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29768 · cited by 0 works (OpenAlex) · open-access PDF

Hindi

दृष्टि-भाषा संवेदना दृष्टि-भाषा संवेदनाओं के साथ प्राकृतिक भाषा को संरेखित करने में प्रभावशाली प्रगति हुई है, फिर भी उच्च-स्तरीय संवेदनाओं को पार-स्तरीय भौतिक बातचीत में बुनियादी बनाना अभी भी चुनौतीपूर्ण है। इस अंतर को संबोधित करने के लिए, हम UniRAP की पेशकश करते हैं, जो एक संयुक्त इंटरफ़ेस टोकन तंत्र के माध्यम से पार-स्तरीय भौतिक संवेदनाओं को अनुमानित करने और संचालन योग्य भौगोलिक प्रतिनिधित्वों के लिए संदर्भ भाषा निर्देशों को मानचित्रित करने के लिए एक सामंजस्यपूर्ण मॉडल है। UniRAP इस समस्या को एक शर्तित बहुमूल्य जनरेख कार्य के रूप में परिभाषित करता है, दृ

Machine translation (Meta M2M100); the English original follows.

Abstract

Vision-language perception has achieved impressive progress in aligning natural language with visual observations, yet grounding high-level semantics into part-level physical interaction remains challenging. To address this gap, we propose UniRAP, a unified model for inferring part-level physical affordances and mapping language instructions to actionable geometric representations. UniRAP formulates this problem as a conditional multimodal generation task, integrating visual inputs, textual instructions, and optional prompts into a shared spatiotemporal representation space through a unified interface token mechanism. To predict executable contact geometry, we introduce a Unified Affordance Decoder (UAD), which jointly performs object detection, part-level affordance segmentation, and 4-DoF interaction pose estimation by leveraging intermediate segmentation features. In addition, we propose a curriculum-based transfer training strategy that progressively adapts the model from general visual parsing to interaction-aware perception, improving data efficiency under limited textual annotations. Experiments show that UniRAP achieves state-of-the-art performance on referring expression segmentation, affordance grounding, and interaction pose estimation, while maintaining strong spatiotemporal consistency in dynamic video scenarios. These results demonstrate the effectiveness of UniRAP as a unified perception framework for language-guided physical manipulation.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Affordance, Computer science, Executable, Human–computer interaction, Artificial intelligence, Semantics (computer science)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange