Newest papers

How Depth Growth and Looping Shape In-Context Learning in Language Models

Ferdinand Kapl, Kaitlin Maile, Stefan Bauer, Johannes von Oswald · 2026 · arXiv

DOI: 10.48550/arXiv.2610.31421 · open-access PDF

Japanese

ロープで深く成長した言語モデルは、深い計算を再利用し、その推論能力に興味を持ちました。より良いインコンテキスト学習(ICL)の事前の示唆的な証拠に基づいて、このような再利用がICLをどのように好むかを調査し、また、この能力の取得、保持、およびメカニズムをどのように形作るかを含む。私たちの分析は3Bパラメータおよび制御されたタスクまで前訓練された言語モデルをカバーしています。

Machine translation (Meta M2M100); the English original follows.

Abstract

Looped and depth-grown language models reuse computations across depth and have attracted interest for their reasoning capabilities. Building on prior suggestive evidence of improved in-context-learning (ICL), we investigate how such reuse favors ICL, including how it shapes the acquisition, retention, and mechanisms of this ability. Our analysis spans pretrained language models up to 3B parameters and controlled tasks. The experiments provide evidence for a shared inductive bias toward learning from context: grown and looped models rely more on information in context and, compared with a full

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange