Liana Patel, Rishabh Ranjan, Ion Stoica, Matei Zaharia, Carlos Guestrin · 2026 · arXiv
DOI: 10.48550/arXiv.2610.26754 · open-access PDF
Pembaca manusia ahli memproses input besar (misalnya, repositori kode dan dokumen panjang) dengan skimming untuk membaca hanya apa yang penting. Di sisi lain, model bahasa memproses semua token input melalui setiap lapisan model, mengakibatkan biaya pemrosesan yang signifikan untuk input konteks panjang. Kami memperkenalkan TokenGate, komponen kecil, drop-in yang memperluas model terlatih yang ada untuk memungkinkan pemotongan token yang dapat dipelajari dari ujung ke ujung melalui input konteks panjang untuk prefill yang cepat dan memori efisien.
Machine translation (Meta M2M100); the English original follows.
Expert human readers process large inputs (e.g., code repositories and long documents) by skimming to closely read only what matters. On the other hand, language models process all input tokens through each model layer, resulting in significant processing costs for long-context inputs. We introduce TokenGate, a small, drop-in component that extends existing trained models to enable end-to-end learnable token pruning over long-context inputs for fast and memory-efficient prefill. To enable end-to-end learning despite the non-differentiable top-K operator for token pruning, we introduce token-ga
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26754, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange