Jakob Steimle, Thomas Elsken, Daniel Ehrhardt, Lukas Rinder, Daniel Cremers, Xi Wang · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28369 · open-access PDF
越来越多的研究人员为Mixure-of-Experts (MoE) 模型提出了动态的专家跳跃方法,这些模型调整了推断过程中每个代币激活的专家数量,旨在保持模型质量,同时进一步减少活跃参数的数量。这些方法增加了校准的重要性分数,专家间相似度量,或学习每代币分配器,以便识别专家跳过。
Machine translation (Meta M2M100); the English original follows.
A growing body of work proposes dynamic expert-skipping methods for Mixture-of-Experts (MoE) models that adjust the number of experts activated per token during inference, aiming to preserve model quality while further reducing the number of active parameters. These methods have added calibrated importance scores, inter-expert similarity metrics, or learned per-token allocators in order to identify experts to skip. However, different calibration regimes, different reductions in the effective number of experts, and evaluation suites make comparisons difficult. We classify four representative pr
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28369, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange