Newest papers

When the harness decides success: state that cannot be re-observed

Raimondas Lencevicius · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26632 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

这些选项决定一个语言模型在轮回中看到的内容:他们放下旧轮回,总结它们,或明智地重复模型本身的早期推理。我们问,当这些选择改变一个代理人是否成功,而不仅是成本,并报告三项发现时,首先,在预注册的合成探测器中,在低成本推理模型上提前注册的重复成本没有什么成功,当需要的事实再次出现时(相反,事实被失去),但在事实无法重新观察时,它将成功分离为机会:两项罚款差异为92,5百分点(90% CI [89.5, 95.5])。在第二个提供商的模型上预订的重复成功时,无法显示相同的对比。一旦事实被失去,该模型没有选择相同的答案,

Machine translation (Meta M2M100); the English original follows.

Abstract

Agent harnesses decide what a language model sees across turns: they drop old turns, summarize them, or replay the model's own earlier reasoning. We ask when these choices change whether an agent succeeds, not only what it costs, and report three findings. First, in a pre-registered synthetic probe, truncating the context window costs a low-cost reasoning model nothing in success when a needed fact can be looked up again (it looks the fact up instead), but drops its success to chance when the fact cannot be re-observed: the two penalties differ by 92.5 percentage points (90% CI [89.5, 95.5]). A pre-registered replication on one model from a second provider showed the same contrast. Once the fact was lost, that model chose no answer rather than guessing. Second, on ARC-AGI-3, where the benchmark's Standard harness trails its Provider Adapter harness by a wide margin, we decompose that gap one component at a time, for one model on four games. Summarizing old turns instead of dropping them, while keeping the Standard harness's invitation to take notes, recovers 86% of a 63-point gap without replaying the model's reasoning. The same harness without the invitation recovers 43%, and adding the provider's native replay of the model's reasoning recovers the remaining 57%. These are point estimates from one counted run per configuration and game, and the replay share depends on a single run. Third, in the benchmark's published runs and in ours, the Adapter's advantage in the probability of clearing a level it has reached is larger at deep levels than at early ones. This largely restates where runs stopped and does not separate a growing advantage from a constant one.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Context (archaeology), Adapter (computing), Ask price, Clearing, Nothing

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange