Yifan Yang, Hanling Yi, Dongquan Yang, Feng Lin, Rong Xiao, Aijun Li, Xiaotian Yu · 2026 · arXiv
DOI: 10.48550/arXiv.2610.29662 · open-access PDF
混合专家(MoE)架构高效地缩小模型容量,但它们的大量内存足迹构成了部署的主要障碍。专家切割减少了这种足迹,但大多数方法都专注于专家如何平等地删除和切割每个层。我们建议SPARE(通过路由 Entropy进行散点分配),这是一个无训练的,选择器-agnostic 方法,决定使用路由 Entropy从每个层中删除多少专家,代币的规范化 Top-K 网关对校准数据产生重量。
Machine translation (Meta M2M100); the English original follows.
Mixture-of-Experts (MoE) architectures scale model capacity efficiently, yet their substantial memory footprint poses a major barrier to deployment. Expert pruning reduces this footprint, but most methods focus on which experts to remove and prune every layer equally. We propose SPARE (Sparsity Allocation via Routing Entropy), a training-free, selector-agnostic method that decides how many experts to remove from each layer using routing entropy, the entropy of tokens' normalized Top-K gate weights on calibration data. When a token's gate weight is spread evenly, removing one active expert disc
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.29662, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange