Yuzhou Hong · 2026 · arXiv
DOI: 10.48550/arXiv.2610.28540 · open-access PDF
Model bahasa autoregressive dilatih oleh cross-entropy token berikutnya, namun beberapa sistem baru-baru ini juga mengulangi keadaan tersembunyi tanpa mengeluarkan token: pikiran berkelanjutan, langkah-langkah pause, dan kedalaman berulang. Catatan ini memisahkan tiga risiko pada satu proses tersembunyi. Risiko token $R_{\mathrm{tok}}$ mencetak pembacaan. Risiko tersembunyi $R_1$ mencetak peta satu langkah yang melihat keadaan tersembunyi yang sebenarnya. Risiko lingkaran terbuka $R_H$ mencetak peta yang sama setelah itu telah diulangi pada outputnya sendiri. Jika token Bayes berikutnya di belakang adalah fungsi dari statistik $k$-dimensional dengan covaria penuh-rang
Machine translation (Meta M2M100); the English original follows.
Autoregressive language models are trained by next-token cross-entropy, yet several recent systems also iterate a hidden state without emitting a token: continuous thoughts, pause steps, and recurrent depth. This note separates three risks on one latent process. The token risk $R_{\mathrm{tok}}$ scores a readout. The teacher-forced latent risk $R_1$ scores a one-step map that sees the true hidden state. The open-loop risk $R_H$ scores the same map after it has been iterated on its own outputs. If the Bayes next-token posterior is a function of a $k$-dimensional statistic with full-rank covaria
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.28540, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange