Newest papers

CertMoE: Robustness Certification for Mixture-of-Experts LLMs via Routing-Aware Smoothing

Qingyue Wang, Yudong Gao, Lixia Wang, Ruixuan Huang, Yuanyuan Yuan, Yangqiu Song, Shuai Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29395 · cited by 0 works (OpenAlex) · open-access PDF

French

Les cadres de certification existants, inspirés par l'interprétation abstraite ou l'exécution symbolique, prédisent généralement un graphique de calcul fixe, où les perturbations d'entrée ou les variations n'affectent que les sorties de modèle, mais pas le chemin d'exécution sous-jacent. Cette hypothèse est cependant violée dans les modèles de langue abstraite ou d'exécution symbolique, dont le routage dépendant de l'entrée induit généralement des voies d'exécution discrètes et dynamiques. Crucialement, ce calcul dynamique change l'objet certifié d'une seule fonction fixe à une composition de sous-fonctions d'exécution dépendantes de l'

Machine translation (Meta M2M100); the English original follows.

Abstract

Provable robustness is a cornerstone of trustworthy machine learning, aiming to guarantee model predictions under adversarial perturbations. Existing certification frameworks, inspired by abstract interpretation or symbolic execution, typically assume a fixed computation graph, where input perturbations or variations affect only model outputs but not the underlying execution path. This assumption, however, is violated in Mixture-of-Experts (MoE) Large Language Models (LLMs), whose input-dependent routing induces discrete and dynamic execution paths. Crucially, this dynamic computation changes the certified object from a single fixed function to a composition of input-dependent sub-functions, leading to unreliable certification guarantees. This paper presents \textsc{CertMoE}, the first routing-aware robustness certification framework for MoE LLMs, which explicitly accounts for input-induced routing dynamics in certification. Specifically, we introduce a relaxed routing-containment criterion, which certifies that perturbed Top-$K$ routing is contained within a larger certified expert pool. Based on this guarantee, \textsc{CertMoE} then constructs the corresponding induced sub-network and certifies its prediction consistency. This hierarchical certification procedure yields a joint probabilistic guarantee over routing containment and sub-network robustness, providing sufficient conditions under which the original prediction remains invariant. Extensive experiments across recent MoE LLMs demonstrate that dynamic routing makes certificates issued by existing methods frequently violated, while \textsc{CertMoE} reduces the empirical violation rate with routing-aware guarantees, e.g., from more than $30\%$ to $0$, yielding certificates that more faithfully reflect the robustness of MoE models.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Robustness (evolution), Certification, Computer science, Probabilistic logic, Computation, Routing (electronic design automation)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange