Newest papers

VisualEvolve: A Skill Self-Evolution Framework for Visual Reasoning

Jiawei Wang, Yucun Li, Xueqi Li, Shenghao Wang, Xinquan Gong, Dan Zhang, Zheqi Lv, Yuquan Le, Shaozu Yuan, An Zhang, Da Cao, Tat-Seng Chua · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28386 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Penggunaan alat-alat visual yang efektif secara langsung memungkinkan model data Trawer-language (VLM) untuk memperoleh bukti-bukti yang relevan dengan tugas, tetapi efektivitasnya dapat dibatasi oleh operasi visual yang ditentukan sebelumnya dan strategi-strategi penggunaan alat-alat visual yang dipelajari selama pelatihan. Menampilkan operasi dan strategi ini sebagai keterampilan eksternal yang dapat digunakan kembali memungkinkan mereka untuk disempurnakan melalui evolusi diri tanpa memperbarui parameter model. Namun, penyempurnaan keterampilan dapat memperkenalkan regresi kinerja untuk VLM. Sementara itu, memeriksa jalur eksekusi tugas yang terakumulasi dari siklus Evaluasi bersama dengan sejarah evolusi keterampilan dapat menyebabkan overload bukti evolusi. Untuk mengatasi tantangan ini, kami mengusulkan VisualEvolve, sebuah kerangka kerja yang mengembangkan diri sendiri melalui evolusi tanpa memperbarui parameter model.

Machine translation (Meta M2M100); the English original follows.

Abstract

Visual tool use enables vision-language models (VLMs) to acquire task-relevant evidence, but its effectiveness can be limited by predefined visual operations and the tool-use strategies learned during training. Representing these operations and strategies as reusable external skills allows them to be refined through self-evolution without updating model parameters. However, skill refinement can introduce performance regressions. Meanwhile, inspecting accumulated task execution trajectories together with skill evolution history may lead to evolution evidence overload. To address these challenges, we propose VisualEvolve, a self-evolving framework that evolves executable visual operations and tool-use strategies for VLMs through a Select--Execute--Evolve--Evaluate cycle. Skill-Library Snapshot Selection uses performance on a fixed validation set disjoint from execution tasks to select an effective historical snapshot for subsequent evolution, limiting the propagation of performance regressions. Progressive Evidence Access (PEA) guides capability-gap diagnosis through a coarse-to-fine hierarchy comprising the Skill-Performance Report, Evolution Log, Trajectory Summaries, and relevant raw trajectory segments. Across four VLM backbones and six evaluation sets from three benchmarks, VisualEvolve improves over the Initial Skill Library in 23 of 24 settings, with a maximum gain of 15.1 percentage points on VisualProbe Hard using Qwen3.5-Plus. Ablation studies support the contributions of Snapshot Selection and PEA, while further experiments show that skills can be improved using experience from different data settings and directly reused across backbones. Moreover, VisualEvolve enables effective skill self-evolution with over an order of magnitude fewer task-solving trajectories than estimated for representative RL-based methods.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Snapshot (computer storage), Executable, Artificial intelligence, Machine learning, Visual hull

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange