Newest papers

A Fisher-Geometric View of Multi-Teacher On-Policy Distillation

Liu Kaiyuan, Zhuang Ziyuan, Ma Haoxuan, Yannian Gu, Song Mingfei, Rongxiang Weng, Ye Jieping · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26525 · cited by 0 works (OpenAlex) · open-access PDF

French

Consolidation de ces modèles de fondation spécialisés de domaine ouvert dans un généraliste unifié est un objectif central de la formation post-formation. Deux approches sont en usage commun: la fusion de modèle, qui est rapide mais traitée comme une tâche heuristique, et la distillation spécifique sur la politique impliquée de plusieurs enseignants, qui correspond aux distributions mais exige l'objectif d'isolement lourd de la fusion de Fisher et souffre du conflit de gradient entre les tâches. Nous montrons que les deux sont un objectif vu à des ordres différents. Dans un bassin partagé de spécialistes homologues, l'objectif de distillation spécifique sur la politique impliquée de plusieurs enseignants est, en second ordre, exactement l'objectif de

Machine translation (Meta M2M100); the English original follows.

Abstract

Consolidating specialized foundation models into a unified generalist is a central goal of post-training. Two approaches are in common use: model merging, which is fast but treated as a heuristic, and multi-teacher on-policy distillation, which matches distributions but demands heavy rollout compute and suffers from cross-task gradient conflict. We show that the two are one objective seen at different orders. Within a shared basin of homologous specialists, the multi-teacher on-policy distillation objective is, to second order, exactly the Fisher-merging objective. Its minimizer is therefore the Fisher merge, a classic model-merging method given by a closed-form combination of the specialists, so merging is optimal, to second order, as a closed-form stage of distillation. Building on this identification, we propose Residual-Governed Branch-and-Merge(RGBM), a two-phase consolidation procedure. Concretely, a full-parameter Fisher merge is intractable on foundation models, so Phase~1 restricts the merge to the task-vector subspace. Extending tangent-space analyses of merging, we show that the same coefficients that mix task vectors also mix implicit rewards, which lets a residual program on unlabelled prompts solve for them. The program minimizes the worst per-domain residual and reports its energy-normalized values as the domain residual spectrum. Phase~2 lets this spectrum govern what may be added: RGBM passivates domains with low normalized residuals, trains isolated branches only for domains above the spectral gap, and writes back the admitted updates with calibrated shrinkage. On two open-source consolidation suites, the Phase~1 merge alone matches or outperforms fully trained multi-teacher distillation, and RGBM improves on it further with zero extra joint-training steps, whereas every un-gated write-back, including continued joint distillation from the merge, falls below it.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Merge (version control), Residual, Distillation, Computer science, Mathematics, Algorithm

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange