Newest papers

VisualEvolve: A Skill Self-Evolution Framework for Visual Reasoning

Jiawei Wang, Yucun Li, Xueqi Li, Shenghao Wang, Xinquan Gong, Dan Zhang, Zheqi Lv, Yuquan Le, Shaozu Yuan, An Zhang, Da Cao, Tat-Seng Chua · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28386 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

通过视觉工具的有效操作,可以直接将视觉语言数据模型(VLMs)收集到相关任务的证据,但其有效性可以受到预定义的视觉操作和从训练中学到的工具使用策略的限制。将这些操作和策略作为可重复使用的外部技能呈现出来,可以通过自我进化而改进,而不会更新模型参数。然而,技能改进可以引入性能回归。同时,检查积累的任务执行轨迹和技能进化历史可能导致进化证据过载。 为了应对这些挑战,我们建议VisualEvolve,一个自我进化框架,通过使用可执行的视觉操作和工具使用的VLM的最大策略进化。 通过选择 - 执行 - 进化 - 评估 - 评估周期。 技能 - 图

Machine translation (Meta M2M100); the English original follows.

Abstract

Visual tool use enables vision-language models (VLMs) to acquire task-relevant evidence, but its effectiveness can be limited by predefined visual operations and the tool-use strategies learned during training. Representing these operations and strategies as reusable external skills allows them to be refined through self-evolution without updating model parameters. However, skill refinement can introduce performance regressions. Meanwhile, inspecting accumulated task execution trajectories together with skill evolution history may lead to evolution evidence overload. To address these challenges, we propose VisualEvolve, a self-evolving framework that evolves executable visual operations and tool-use strategies for VLMs through a Select--Execute--Evolve--Evaluate cycle. Skill-Library Snapshot Selection uses performance on a fixed validation set disjoint from execution tasks to select an effective historical snapshot for subsequent evolution, limiting the propagation of performance regressions. Progressive Evidence Access (PEA) guides capability-gap diagnosis through a coarse-to-fine hierarchy comprising the Skill-Performance Report, Evolution Log, Trajectory Summaries, and relevant raw trajectory segments. Across four VLM backbones and six evaluation sets from three benchmarks, VisualEvolve improves over the Initial Skill Library in 23 of 24 settings, with a maximum gain of 15.1 percentage points on VisualProbe Hard using Qwen3.5-Plus. Ablation studies support the contributions of Snapshot Selection and PEA, while further experiments show that skills can be improved using experience from different data settings and directly reused across backbones. Moreover, VisualEvolve enables effective skill self-evolution with over an order of magnitude fewer task-solving trajectories than estimated for representative RL-based methods.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Snapshot (computer storage), Executable, Artificial intelligence, Machine learning, Visual hull

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange