Newest papers

VisualEvolve: A Skill Self-Evolution Framework for Visual Reasoning

Jiawei Wang, Yucun Li, Xueqi Li, Shenghao Wang, Xinquan Gong, Dan Zhang, Zheqi Lv, Yuquan Le, Shaozu Yuan, An Zhang, Da Cao, Tat-Seng Chua · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28386 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

ビジュアルツールの有効な操作は、トレイヤー言語のデータモデル(VLMs)を直接使用して、タスク関連の証拠を取得することを可能にしますが、その有効性は、事前定義されたビジュアル操作とツールの使用戦略によって制限されます。これらの操作と戦略を再利用可能な外部スキルとして表すことは、タスク関連のパラメータを更新することなく、自己進化を通じて精進するフレームワークであるVisualEvolveを提案します。しかし、スキル改良はパフォーマンス回帰を導入することができます。同時に、スキル進化の歴史と一緒に累積したタスク実行軌道を調べることは、進化の証拠の過剰負荷につながります。これらの課題に対処する

Machine translation (Meta M2M100); the English original follows.

Abstract

Visual tool use enables vision-language models (VLMs) to acquire task-relevant evidence, but its effectiveness can be limited by predefined visual operations and the tool-use strategies learned during training. Representing these operations and strategies as reusable external skills allows them to be refined through self-evolution without updating model parameters. However, skill refinement can introduce performance regressions. Meanwhile, inspecting accumulated task execution trajectories together with skill evolution history may lead to evolution evidence overload. To address these challenges, we propose VisualEvolve, a self-evolving framework that evolves executable visual operations and tool-use strategies for VLMs through a Select--Execute--Evolve--Evaluate cycle. Skill-Library Snapshot Selection uses performance on a fixed validation set disjoint from execution tasks to select an effective historical snapshot for subsequent evolution, limiting the propagation of performance regressions. Progressive Evidence Access (PEA) guides capability-gap diagnosis through a coarse-to-fine hierarchy comprising the Skill-Performance Report, Evolution Log, Trajectory Summaries, and relevant raw trajectory segments. Across four VLM backbones and six evaluation sets from three benchmarks, VisualEvolve improves over the Initial Skill Library in 23 of 24 settings, with a maximum gain of 15.1 percentage points on VisualProbe Hard using Qwen3.5-Plus. Ablation studies support the contributions of Snapshot Selection and PEA, while further experiments show that skills can be improved using experience from different data settings and directly reused across backbones. Moreover, VisualEvolve enables effective skill self-evolution with over an order of magnitude fewer task-solving trajectories than estimated for representative RL-based methods.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Snapshot (computer storage), Executable, Artificial intelligence, Machine learning, Visual hull

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange