Newest papers

CertMoE: Robustness Certification for Mixture-of-Experts LLMs via Routing-Aware Smoothing

Qingyue Wang, Yudong Gao, Lixia Wang, Ruixuan Huang, Yuanyuan Yuan, Yangqiu Song, Shuai Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29395 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

La robustez de la probabilidad reciente es una piedra angular del aprendizaje de máquina confiable, con el objetivo de garantizar predicciones de modelos bajo perturbaciones adversarias. Los marcos de certificación existentes, inspirados por la interpretación abstracta o la ejecución simbólica, suelen asumir un gráfico de cálculo fijo, en el que las perturbaciones de entrada o las variaciones afectan sólo a la salida del modelo, pero no a la vía de ejecución subyacente. Esta suposición, sin embargo, se viola en los modelos de lenguaje abstracto o simbólico, cuyo enrutamiento dependiente de la entrada induce caminos de ejecución discretos y dinámicos. Crucialmente, este cálculo dinámico cambia el objeto certificado de una función fija a una composición de

Machine translation (Meta M2M100); the English original follows.

Abstract

Provable robustness is a cornerstone of trustworthy machine learning, aiming to guarantee model predictions under adversarial perturbations. Existing certification frameworks, inspired by abstract interpretation or symbolic execution, typically assume a fixed computation graph, where input perturbations or variations affect only model outputs but not the underlying execution path. This assumption, however, is violated in Mixture-of-Experts (MoE) Large Language Models (LLMs), whose input-dependent routing induces discrete and dynamic execution paths. Crucially, this dynamic computation changes the certified object from a single fixed function to a composition of input-dependent sub-functions, leading to unreliable certification guarantees. This paper presents \textsc{CertMoE}, the first routing-aware robustness certification framework for MoE LLMs, which explicitly accounts for input-induced routing dynamics in certification. Specifically, we introduce a relaxed routing-containment criterion, which certifies that perturbed Top-$K$ routing is contained within a larger certified expert pool. Based on this guarantee, \textsc{CertMoE} then constructs the corresponding induced sub-network and certifies its prediction consistency. This hierarchical certification procedure yields a joint probabilistic guarantee over routing containment and sub-network robustness, providing sufficient conditions under which the original prediction remains invariant. Extensive experiments across recent MoE LLMs demonstrate that dynamic routing makes certificates issued by existing methods frequently violated, while \textsc{CertMoE} reduces the empirical violation rate with routing-aware guarantees, e.g., from more than $30\%$ to $0$, yielding certificates that more faithfully reflect the robustness of MoE models.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Robustness (evolution), Certification, Computer science, Probabilistic logic, Computation, Routing (electronic design automation)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange