Newest papers

A Fisher-Geometric View of Multi-Teacher On-Policy Distillation

Liu Kaiyuan, Zhuang Ziyuan, Ma Haoxuan, Yannian Gu, Song Mingfei, Rongxiang Weng, Ye Jieping · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26525 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Consolidar este modelo de fundación especializado de dominio abierto en un generalista unificado es un objetivo central del post-entrenamiento. Dos enfoques se utilizan comúnmente: la fusión del modelo, que es rápida, pero se trata como una tarea heurística, y la destilación especidual en política de varios maestros, que coincide con las distribuciones, pero exige la computación de rollout pesada y sufre un conflicto de gradientes cruzados. Mostramos que los dos son un objetivo visto en diferentes órdenes de dominio. Dentro de un vaso compartido de especialistas homólogos, el objetivo de destilación especular implicada de varios maestros es, en segundo orden, exactamente el objetivo de fusión de Fisher. Su minimizador es por lo tanto el Fisher, un método clásico de fusión dado por un

Machine translation (Meta M2M100); the English original follows.

Abstract

Consolidating specialized foundation models into a unified generalist is a central goal of post-training. Two approaches are in common use: model merging, which is fast but treated as a heuristic, and multi-teacher on-policy distillation, which matches distributions but demands heavy rollout compute and suffers from cross-task gradient conflict. We show that the two are one objective seen at different orders. Within a shared basin of homologous specialists, the multi-teacher on-policy distillation objective is, to second order, exactly the Fisher-merging objective. Its minimizer is therefore the Fisher merge, a classic model-merging method given by a closed-form combination of the specialists, so merging is optimal, to second order, as a closed-form stage of distillation. Building on this identification, we propose Residual-Governed Branch-and-Merge(RGBM), a two-phase consolidation procedure. Concretely, a full-parameter Fisher merge is intractable on foundation models, so Phase~1 restricts the merge to the task-vector subspace. Extending tangent-space analyses of merging, we show that the same coefficients that mix task vectors also mix implicit rewards, which lets a residual program on unlabelled prompts solve for them. The program minimizes the worst per-domain residual and reports its energy-normalized values as the domain residual spectrum. Phase~2 lets this spectrum govern what may be added: RGBM passivates domains with low normalized residuals, trains isolated branches only for domains above the spectral gap, and writes back the admitted updates with calibrated shrinkage. On two open-source consolidation suites, the Phase~1 merge alone matches or outperforms fully trained multi-teacher distillation, and RGBM improves on it further with zero extra joint-training steps, whereas every un-gated write-back, including continued joint distillation from the merge, falls below it.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Merge (version control), Residual, Distillation, Computer science, Mathematics, Algorithm

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange