Newest papers

RACE: Learning Multimodal Search through Referential Abstraction and Compositional Evidence

Eric Tao, Minhao Sun, Liyan Liu, Zhijing Wu, Xiaohan Yu, Lang Mei, Li Huang, Chao Feng, Jinchao Ma, Sichen Kang, Siyu Mo, Chong Chen · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28382 · cited by 0 works (OpenAlex) · open-access PDF

Korean

훈련 멀티모델 검색 에이전트는 응답을 결정하기 위해 시각적으로 토착 된 검색을 필요로하는 데이터 작업을 필요로합니다. 텍스트 정체성 힌트와 사용되지 않은 표적 증거는 합성 질문에서 단계를 허용하고 시각적 검색 및 증거 통합에 대한 감독을 약화시킵니다. 우리는 RACE를 제안합니다. 이미지 조건화 작업 합성 프레임 워크는 이미지에서 표적 특정 증거에서 응답에 대한 의존성을 구축합니다. Referential Abstraction (RA)는 특정 시각적 참조 또는 묘사된 후보들 사이에서 검색을 통해 해결된 명확한 관계 참조로 정체성을 밝히는 텍스트를 대체합니다. Compositional Evidence (CE) 링크 하나의 이미지에서 여러 단체에 대한 사실을

Machine translation (Meta M2M100); the English original follows.

Abstract

Training multimodal search agents requires tasks that make visually grounded search necessary for determining the answer. Textual identity clues and unused target evidence allow shortcuts in synthetic questions, weakening supervision for visual search and evidence integration. We propose RACE, an image-conditioned task synthesis framework that constructs dependencies from images to targets and from target-specific evidence to answers. Referential Abstraction (RA) replaces identity-revealing text with specific visual references or ambiguous relational references resolved by searching among depicted candidates. Compositional Evidence (CE) links retrieved facts about multiple entities in one image or entities from different images into joint questions whose answers require every designated contribution. Both operations also apply to images found during search, allowing new visual observations to determine subsequent search targets or supply answer evidence. Fine-tuning Qwen3.5-27B on only 12K teacher-generated trajectories from RACE raises average accuracy across ten multimodal benchmarks from 53.5 to 56.0, a relative improvement of approximately 4.7% over the matched base agent. Our code, data, and model will be publicly available.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Abstraction, Computer science, Task (project management), Artificial intelligence, Visual search, Natural language processing

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange