Newest papers

When Thoughts Become Memory: Unifying In-Context Learning and Fine-Tuning

Leon Chlon, Julie Huang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29419 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

我们是第一个在这个网络背景下绘制的摄像头打破了对未见数据是否属于预先训练的模型背景或其重量的认证规则,建立在其信息价值的准确规律的基础上。该值,在持有查询中保存的代码长度,等于在预期中对查询中存储的位数,加上预先修复,减少了对任何模型,建筑,数据集和任务的解码损失。法律的天花板证实从一个测量中选择,并且随时有效的代码测试使其从持有查询中获取,随时被控制的错误。每一个证书在训练过的变压器,TabPFN,一个占用网络和一个立体网络上发布的天花板是正确的。测试验证了TabPFN在X

Machine translation (Meta M2M100); the English original follows.

Abstract

We are the first to give a certified rule for whether unseen data belong in a pretrained model's context or its weights, built on an exact law for their information value. That value, the code length saved on held-out queries, equals in expectation the bits stored about the query, plus a prior repair, minus a decoding loss, for any model, architecture, dataset and task. The law's ceilings certify the choice from one measurement, and an anytime-valid test makes it from held-out queries, with error controlled whenever it stops. Every certificate the ceilings issue on a trained transformer, TabPFN, an occupancy network and a stereo network is correct. The test picks TabPFN over XGBoost and a trained surrogate over an in-context learner on airfoil pressure, calls the better of GPT-2's cache and fine-tune in 61,948 of 64,000 query streams over 32 comparisons, and errs in at most 3.3% of null streams. The law also diagnoses when learning in context helps and when it does not. Deleting part of a context is held-out validation of that part in any model, and adding parts one at a time traces the learning curve in context: one patch that sees a shape feature cuts an occupancy network's code length there a hundredfold, while eight that miss it nearly double it, and one camera that sees the surface beats four in a stereo network. Ranked by this label-free valuation, a kriging learner's three best support cars beat four drawn at random, and on a yes-or-no claim the valuation recovers a published evidence adjudicator, with AUROC 0.91.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Certificate, Artificial intelligence, Context (archaeology), Code (set theory), Cache

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange