Newest papers

Auditing Conformal Prediction for Language Models Under Inference-Time Model Shift

Achref Doula, Max Mühlhäuser · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28937 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Konform prediksi (CP) untuk model bahasa biasanya dievaluasi di bawah protokol yang disesuaikan, di mana kalibrasi dan penyebaran menggunakan konfigurasi inferensi post-training yang sama, seperti mode kuantizasi dan suhu inferensi. Dalam penyebaran, pilihan ini sering berbeda dari yang digunakan pada kalibrasi untuk memenuhi memori atau pembatasan throughput atau kebijakan layanan. Kami berpendapat bahwa ini mengekspos keterbatasan dalam protokol evaluasi CP saat ini untuk model bahasa. Kami mengformalisasi fenomena yang mendasari sebagai pergeseran model inferensi-waktu: perubahan dalam distribusi skor konform yang diinduksi oleh konfigurasi inferensi daripada oleh distribusi input-label, dan secara teoritis karakterisasi bagaimana itu melanggar exchangeability split-conformal bahkan ketika distribusi tugas dan model yang dilatih tidak berubah. Kami kemudian mengevaluasi konsekuensi

Machine translation (Meta M2M100); the English original follows.

Abstract

Conformal prediction (CP) for language models is typically evaluated under a matched protocol, in which calibration and deployment use the same post-training inference configuration, such as quantization mode and inference temperature. In deployment, these choices often differ from those used at calibration to meet memory or throughput constraints or serving policies. We argue that this exposes a limitation in current CP evaluation protocols for language models. We formalize the underlying phenomenon as inference-time model shift: a change in the conformal score distribution induced by inference configuration rather than by the input--label distribution, and theoretically characterize how it breaks split-conformal exchangeability even when the task distribution and trained model are unchanged. We then evaluate its consequences across 8 language models, 4 finite-label benchmarks, and 12 inference configurations. We find that matched evaluation can substantially overstate deployment-time coverage, and that the effect is heterogeneous across models and datasets. We argue that inference configuration should be reported as a first-class experimental axis in CP evaluations for language-model systems.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Inference, Computer science, Fiducial inference, Artificial intelligence, Software deployment, Language model

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange