Liana Patel, Rishabh Ranjan, Ion Stoica, Matei Zaharia, Carlos Guestrin · 2026 · arXiv
DOI: 10.48550/arXiv.2610.26754 · open-access PDF
专家的人类读者通过扫描处理大输入(例如代码存储库和长文档)来仔细读取只有重要的内容。另一方面,语言模型通过每个模型层处理所有输入代币,导致长背景输入的重大处理成本。我们引入了TokenGate,一个小型,放入组件,扩展现有的训练有素的模型,以便在长背景输入上进行终端可学习的代币切割,以实现快速且记忆效率高的预装。
Machine translation (Meta M2M100); the English original follows.
Expert human readers process large inputs (e.g., code repositories and long documents) by skimming to closely read only what matters. On the other hand, language models process all input tokens through each model layer, resulting in significant processing costs for long-context inputs. We introduce TokenGate, a small, drop-in component that extends existing trained models to enable end-to-end learnable token pruning over long-context inputs for fast and memory-efficient prefill. To enable end-to-end learning despite the non-differentiable top-K operator for token pruning, we introduce token-ga
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26754, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange