Newest papers

From Reasoning to Action: Benchmarking Frontier AI Models on LIBERO-10 Robot Manipulation Tasks

May Espinola, Qifan Zhang, Xiang Yu · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30051 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Kemajuan potensial baru-baru ini dalam pengaturan batas AI model telah menunjukkan kemampuan luar biasa dalam pemahaman visual, pertimbangan, perencanaan, dan penggunaan alat, meningkatkan kemungkinan untuk menggunakan model AI umum untuk manipulasi robot tanpa pelatihan spesifik tugas. Namun, kemampuan manipulasi mereka belum secara sistematis dievaluasi pada benchmark robotik standar. Dalam laporan ini, kami menyajikan kerangka kerja manipulasi lingkaran tertutup yang bersatu untuk mengevaluasi model AI batas khusus atau keterampilan manipulasi pra-latihan pada LIBERO-10 benchmark. Kerangka kerja kami memungkinkan model untuk mengendalikan robot secara langsung melalui pengamatan visual, robot proprioception, dan panggilan alat terstruktur, terutama dengan menetapkan posektor akhir target dan perintah gripper, tanpa tugas spesifik fin-tuning atau keterampilan manipulasi pra-latihan. Kami menilai enam model misi eksperimental masa depan

Machine translation (Meta M2M100); the English original follows.

Abstract

Recent advances in frontier AI models have demonstrated remarkable capabilities in visual understanding, reasoning, planning, and tool use, raising the possibility of using general-purpose AI models for robot manipulation without task-specific training. However, their manipulation capabilities have not been systematically evaluated on standardized robotics benchmarks. In this report, we present a unified closed-loop manipulation framework for evaluating frontier AI models on the LIBERO-10 benchmark. Our framework enables models to control a robot directly through visual observations, robot proprioception, and structured tool calls, primarily by specifying target end-effector poses and gripper commands, without task-specific fine-tuning or pretrained manipulation skills. We evaluate six frontier AI models, including GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Claude Opus 5, Claude Opus 5.5, and Gemini-4 Argon, across three reasoning effort levels, resulting in 18 configurations and 1,800 evaluation episodes. Our results show that GPT-6 Astra achieves the highest success rate of 44%, followed by Gemini-4 Argon at 43%. Increasing reasoning effort generally improves manipulation performance, but the benefits vary across models and come at additional inference cost. We also observe substantial differences across tasks, with models performing relatively well on object placement tasks but struggling with precise spatial arrangements and complex multi-step interactions. These findings demonstrate both the potential and limitations of frontier AI models for robot manipulation, highlighting the challenges of translating reasoning capabilities into reliable physical actions. We release our evaluation framework and experimental results to support reproducible benchmarking and future research on reasoning-based robot manipulation.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Benchmarking, Artificial intelligence, Robot, Computer science, Robotics, Inference

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange