Newest papers

Multimodal Embodied Agents: A Unified Capability-centric Perspective from Computer-Use to Robot-Use

Yanzhe Chen, Ziyi Yang, Jifeng Zhu, Qiming Huang, R. An, Peiyao Xu, Hesen Yang, Runda Liu, Chang Gong, Zhijun Cao, Zechen Bai, Wenzheng Zeng, Yiqi Lin, Guoqiang Liang, Kevin Ma, Kevin Qinghong Lin, Mike Zheng Shou · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30108 · cited by 0 works (OpenAlex) · open-access PDF

Korean

멀티모델 에이전트 (MMAs)는 논리, 메모리, 도구 및 피드백을 통해 상호 작용을 유지하며, 가장 눈에 띄게 컴퓨터 사용 에이전트로, 로봇 시스템 (RSs)는 물리적 역학 하에서 작업 감지 및 액션을 쌍으로 유지합니다. 우리는 멀티모델 인체 에이전트 (MMEAs)를 목표 지향 시스템으로 정의하여 논리, 메모델 작업과 물리적 행동을 쌍으로하고 결정을 수정합니다. 이것은 중앙 질문을 제기합니다 : 멀티모델 에이전트가 컴퓨터 사용에서 로봇 사용으로 이동할 때 어떤 변화가 발생합니까? 우리는 PAPAV, 다섯 가지 반복적 인 기능의 통일화 가능성 캡-센트릭 프레임 워크를 소개합니다 : 현재

Machine translation (Meta M2M100); the English original follows.

Abstract

Multimodal agents (MMAs) sustain interaction through reasoning, memory, tools, and feedback, most visibly as computer-use agents, while robotic systems (RSs) couple sensing and actuation under physical dynamics. We define multimodal embodied agents (MMEAs) as goal-directed systems that couple multimodal task reasoning with physical action and revise decisions from the resulting feedback. This raises a central question: what changes when a multimodal agent moves from computer-use to robot-use? We introduce PAPAV, a unified capability-centric framework of five recurring functions: Perceive the current state, Anticipate action effects, Plan a feasible course, Act through an interface or body, and Verify the outcome. Defined by function rather than architecture, these capabilities let PAPAV identify what all three share and then compare MMEAs with MMAs and with RSs. Five physical constraints, from partial observability to unverifiable outcomes, leave fewer choices fixed in advance and less room to reverse errors. Across 64 benchmarks, explicit evaluation centers on Act; only 5 assess Anticipate and 3 assess Verify, leaving both largely hidden behind task success. These constraints also frame the open challenges we identify. PAPAV therefore provides a common basis for designing reliable physical agents and measuring progress across all five capabilities rather than task success alone.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Embodied cognition, Task (project management), Perspective (graphical), Action (physics), Computer science, Observability

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange