Newest papers

EvaOmni: An Evidence-Grounded Benchmark for Audio-Visual Reasoning

Xiaoxing Hu, Wenxi Yao, Chaoyuan Shi, Ziqi Ye, xiaorong guo, Peixuan Li, Zhuoran Sun, Ziyang Gong, Peiyuan Zhang, Mingxin Liu, Shuran Ma, Hongbo Liu, Xiaomin Yu, Yu Tian, Wentao Wang, Zhenjie Yang, Jian Hu, Yunhang Shen, Haoyu Cao, Xue Yang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28444 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

Omni LLMsは自然に重要なオーディオおよびビジュアル証拠を見つけることができ、オーディオビジュアル推論のための方法を通じて情報を統合することもできます。これらの既存のオーディオビジュアル推論基準は、主に複数の選択の質問に依存し、通常、レベル評価基準の信頼性をさらに弱体化させます。これらの問題に対処するには、モデルが真に証拠に基づいた推論基準であるEvaOmniを導入し、証拠から正しい答えに含まれる392のビデオと1027の純粋に人間が指摘した質問と答えの証拠のカップルを含む。さらに、これらのテキストカップルは、必要なオーディオビジュアル証拠を同時に分析することなく正

Machine translation (Meta M2M100); the English original follows.

Abstract

Omni LLMs can naturally locate key audio and visual evidence and integrate information across modalities for audiovisual reasoning. Existing audio-visual reasoning benchmarks that mainly rely on multiple-choice questions usually score models solely based on answer correctness, making it difficult to assess whether a model truly forms a reasoning chain from evidence to the correct answer. In addition, textual or unimodal shortcuts may allow models to answer correctly without analyzing the necessary audio-visual evidence, further weakening the reliability of evaluation benchmarks. To address these issues, we introduce EvaOmni, an evidence-grounded benchmark containing 392 videos and 1,027 purely human-annotated question-answer pairs, covering 15 capabilities and three levels: retrieval and localization, basic understanding, and complex reasoning. We design a series of rigorous annotation procedures in which experts annotate audio and visual evidence and construct video-based distractors; humans review whether questions meet strict inspection criteria, and finally models filter textual and unimodal shortcuts. Beyond high-quality question-answer pairs, we require models to simultaneously return the answer and supporting evidence intervals in order to evaluate the complete reasoning chain. We propose the Coverage Grounded Score (C.G.) to measure how much annotated evidence a model identifies, and the Bottleneck Grounded Score (B.G.) to penalize poor temporal localization in either modality; both are incorporated into answer correctness. Across 7 open-weight and 17 proprietary Omni LLMs, Gemini3.7-Flash achieved the highest overall accuracy (79.4%), C.G. (70.8%), and B.G. (35.1%). At Level 3, it reached 79.1% accuracy, but its B.G. was only 12.3%. These results show that even the most advanced models still have room for development in grounded audio-visual reasoning.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Benchmark (surveying), Bottleneck, Visual reasoning, Artificial intelligence, Filter (signal processing)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange