Newest papers

CertMoE: Robustness Certification for Mixture-of-Experts LLMs via Routing-Aware Smoothing

Qingyue Wang, Yudong Gao, Lixia Wang, Ruixuan Huang, Yuanyuan Yuan, Yangqiu Song, Shuai Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29395 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Rangka kerja sertifikasi yang ada, terinspirasi oleh interpretasi abstrak atau eksekusi simbolik, biasanya mengasumsikan grafik komputasi tetap, di mana gangguan input atau variasi hanya mempengaruhi output model tetapi bukan jalur eksekusi yang mendasarinya. Asumsi ini, bagaimanapun, dilanggar dalam Rangka kerja sertifikasi Mixture-of-Experts (MoE) Large Language Models (LLMs), yang routing input-dependent biasanya menginduksi jalur eksekusi diskret dan dinamis. Pada dasarnya, perhitungan dinamis ini mengubah objek yang disertifikasi dari fungsi tunggal yang tetap menjadi komposisi sub-fungsi yang tergantung pada input, yang mengarah ke jaminan sertifikasi yang tidak dapat diandalkan dari hierarchy-routence asli. Dokumen ini menyajikan metode robustness yang lebih besar dari yang kita ketah

Machine translation (Meta M2M100); the English original follows.

Abstract

Provable robustness is a cornerstone of trustworthy machine learning, aiming to guarantee model predictions under adversarial perturbations. Existing certification frameworks, inspired by abstract interpretation or symbolic execution, typically assume a fixed computation graph, where input perturbations or variations affect only model outputs but not the underlying execution path. This assumption, however, is violated in Mixture-of-Experts (MoE) Large Language Models (LLMs), whose input-dependent routing induces discrete and dynamic execution paths. Crucially, this dynamic computation changes the certified object from a single fixed function to a composition of input-dependent sub-functions, leading to unreliable certification guarantees. This paper presents \textsc{CertMoE}, the first routing-aware robustness certification framework for MoE LLMs, which explicitly accounts for input-induced routing dynamics in certification. Specifically, we introduce a relaxed routing-containment criterion, which certifies that perturbed Top-$K$ routing is contained within a larger certified expert pool. Based on this guarantee, \textsc{CertMoE} then constructs the corresponding induced sub-network and certifies its prediction consistency. This hierarchical certification procedure yields a joint probabilistic guarantee over routing containment and sub-network robustness, providing sufficient conditions under which the original prediction remains invariant. Extensive experiments across recent MoE LLMs demonstrate that dynamic routing makes certificates issued by existing methods frequently violated, while \textsc{CertMoE} reduces the empirical violation rate with routing-aware guarantees, e.g., from more than $30\%$ to $0$, yielding certificates that more faithfully reflect the robustness of MoE models.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Robustness (evolution), Certification, Computer science, Probabilistic logic, Computation, Routing (electronic design automation)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange