Newest papers

Is In-Context Reinforcement Learning A Posterior Sampler? An Empirical Study

Weiqin Chen, Xinlong Du, Gesualdo Scutari, Harsha Honnappa · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28977 · cited by 0 works (OpenAlex) · open-access PDF

Korean

ICRL(Decision-Pretrained Transformer)는 대규모 언어 모델의 인상적인 컨텍스트 학습 능력을 활용하여 결론 시간에 새롭고 보이지 않는 과제로 일반화합니다. 많은 접근법 중에서 Decision-Pretrained Transformer(DPT)는 이론적 토대를 위해 특히 매력적입니다: 충분히 표현적인 모델과 무한한 사전 훈련 데이터와 같은 이상화된 가정하에 DPT는 바이에시아 뒤 샘플링을 수행 할 수 있습니다. 이 특성화가 실제로 살아남는지 여부는 열려 있습니다. 따라서 우리는 질문합니다 : \textit{is ICRL, 특히 DPT와 우리의 제안된 DPT 변형, 실습에서 바이에시아 뒤 샘플링 모델?}이 질문에 접근하기 위해 우리는 바이에시아

Machine translation (Meta M2M100); the English original follows.

Abstract

In-context reinforcement learning (ICRL) exploits the impressive in-context learning ability of large language models to zero-shot generalize to new, unseen tasks at inference time without any parameter updates. Among many approaches, the Decision-Pretrained Transformer (DPT) is particularly appealing for its theoretical grounding: under idealized assumptions such as a sufficiently expressive model and infinite pretraining data, DPT can perform Bayesian posterior sampling. Whether this characterization survives in practice remains open. Therefore, we ask: \textit{is ICRL, specifically DPT and our proposed DPT variants, a Bayesian posterior sampler in practice?} To approach this question, we establish two necessary conditions that a Bayesian posterior sampler must satisfy: \textit{sufficient-statistic invariance} and the \textit{martingale property}. We then construct statistical hypothesis tests to statistically assess if practically-trained ICRL models satisfy these conditions. Applying these tests in Gaussian, Bernoulli, and linear bandit settings, we find that both conditions are consistently violated, suggesting that ICRL may not behave as Bayesian posterior samplers in practice. Beyond this negative result, our tests provide a principled toolkit for testing the Bayesian behavior of ICRL.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Bayesian probability, Posterior probability, Machine learning, Bayesian inference, Artificial intelligence, Computer science

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange