Newest papers

RACE: Learning Multimodal Search through Referential Abstraction and Compositional Evidence

Eric Tao, Minhao Sun, Liyan Liu, Zhijing Wu, Xiaohan Yu, Lang Mei, Li Huang, Chao Feng, Jinchao Ma, Sichen Kang, Siyu Mo, Chong Chen · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28382 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

O treinamento de agentes de busca multimodal requer tarefas que tornam a pesquisa visualmente fundamentada necessária para determinar a resposta. Dicas de identidade textual e evidências de alvo não utilizadas permitem atalhos em perguntas sintéticas, enfraquecendo a supervisão para pesquisa visual e integração de evidências. Propomos RACE, uma estrutura de síntese de tarefas condicionada por imagem que constrói dependências de imagens para alvos e de evidências específicas de alvos para respostas. Abstração de referência (RA) substitui o texto revelador de identidade por referências visuais específicas ou referências relacionais ambíguas resolvidas através da pesquisa entre candidatos descritos. Evidência Composicional (CE) liga fatos recuperados sobre múltiplas entidades em uma imagem ou entidades de diferentes imagens para perguntas conjuntas cujas resp

Machine translation (Meta M2M100); the English original follows.

Abstract

Training multimodal search agents requires tasks that make visually grounded search necessary for determining the answer. Textual identity clues and unused target evidence allow shortcuts in synthetic questions, weakening supervision for visual search and evidence integration. We propose RACE, an image-conditioned task synthesis framework that constructs dependencies from images to targets and from target-specific evidence to answers. Referential Abstraction (RA) replaces identity-revealing text with specific visual references or ambiguous relational references resolved by searching among depicted candidates. Compositional Evidence (CE) links retrieved facts about multiple entities in one image or entities from different images into joint questions whose answers require every designated contribution. Both operations also apply to images found during search, allowing new visual observations to determine subsequent search targets or supply answer evidence. Fine-tuning Qwen3.5-27B on only 12K teacher-generated trajectories from RACE raises average accuracy across ten multimodal benchmarks from 53.5 to 56.0, a relative improvement of approximately 4.7% over the matched base agent. Our code, data, and model will be publicly available.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Abstraction, Computer science, Task (project management), Artificial intelligence, Visual search, Natural language processing

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange