Advait Gupta, Sachin Konan, Tom Yin, Gabriel Sarch, Zhuang Liu · 2026 · arXiv
DOI: 10.48550/arXiv.2610.31494 · open-access PDF
Meskipun evaluasi online dalam simulator dan lingkungan dunia nyata sangat penting untuk mengukur kinerja agen interaktif, mereka mahal dan menantang untuk skala. benchmarks VQA offline tambahan dapat memungkinkan perbandingan yang lebih cepat dan lebih dapat diandalkan di berbagai domain agen, namun benchmarks yang ada tetap terbatas dan terfragmentasi. Untuk mengatasi kesenjangan ini, kami memperkenalkan AgentVQA, benchmark offline untuk mengevaluasi VLM pada agen realistis
Machine translation (Meta M2M100); the English original follows.
Vision-language models (VLMs) can perform a broad range of tasks across diverse settings, yet their performance in agentic contexts remains poorly understood. Although online evaluations in simulators and real-world environments are essential for measuring interactive agent performance, they are costly and challenging to scale. Complementary offline VQA benchmarks can enable faster and more reliable comparisons across agentic domains, yet existing benchmarks remain limited and fragmented. To address this gap, we introduce AgentVQA, an offline benchmark for evaluating VLMs on realistic agentic
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.31494, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange