Ferdinand Kapl, Kaitlin Maile, Stefan Bauer, Johannes von Oswald · 2026 · arXiv
DOI: 10.48550/arXiv.2610.31421 · open-access PDF
둥글고 깊이 있는 언어 모델은 깊이에 걸쳐 계산을 재사용하고 그들의 추론 능력에 대한 관심을 끌었습니다. 개선된 컨텍스트 내 학습 (ICL)에 대한 이전의 제안적인 증거를 바탕으로, 우리는 이러한 재사용이 ICL을 어떻게 유리하게 만드는지, 이 능력의 습득, 보존 및 메커니즘을 형성하는 방법을 포함하여 조사합니다. 우리의 분석은 3B 매개 변수와 제어된 작업까지의 사전 훈련 된 언어 모델을 다룹니다.
Machine translation (Meta M2M100); the English original follows.
Looped and depth-grown language models reuse computations across depth and have attracted interest for their reasoning capabilities. Building on prior suggestive evidence of improved in-context-learning (ICL), we investigate how such reuse favors ICL, including how it shapes the acquisition, retention, and mechanisms of this ability. Our analysis spans pretrained language models up to 3B parameters and controlled tasks. The experiments provide evidence for a shared inductive bias toward learning from context: grown and looped models rely more on information in context and, compared with a full
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.31421, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange