Newest papers

OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation

Qi Li, Xinchao Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31616 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Pada dasarnya, distilasi grafis telah digunakan secara luas untuk meringankan ketidaksesuaian distribusi distilasi grafis off-policy. Self-distilasi efektif lebih lanjut menghilangkan kebutuhan untuk seorang guru eksternal dengan menggunakan output siswa sendiri, ditingkatkan dengan konteks tambahan atau demonstrasi, sebagai pengawasan. Desain gratis guru ini membuat distilasi grafis sendiri praktis, namun risiko privasinya tetap sebagian besar belum dijelajahi. Secara khusus, menggunakan dua pandangan gabungan dari contoh pelatihan yang sama dapat memperkuat memori. Termotivasi oleh ini, kami menyajikan studi pertama dari serangan inferensi keanggotaan terhadap self-distilasi on-policy. Kami menyelidiki kedua model ancaman kotak abu-abu dan kotak hitam dan mengusulkan kerangka serangan tunggal, yang disebut OP-Print. Acint-Print membangun pada temuan kami yang sama bahwa tanda tangan

Machine translation (Meta M2M100); the English original follows.

Abstract

On-policy distillation has been widely adopted for alleviating the distribution mismatch of off-policy distillation. Self-distillation further removes the need for an external teacher by using the student's own outputs, enhanced with extra context or demonstrations, as supervision. This teacher-free design makes on-policy self-distillation practical, yet its privacy risks remain largely unexplored. In particular, using two coupled views of the same training example may strengthen its memorization. Motivated by this, we present the first study of membership inference attacks against on-policy self-distillation. We investigate both gray-box and black-box threat models and propose a unified attack framework, named OP-Print. OP-Print builds on our finding that the supervision imprint in on-policy self-distillation is sparse and structured, concentrating on uncertainty-sensitive correction sites rather than across the entire rollout. Exploiting this property, we construct attack signals tailored to the two access levels: in the gray-box setting, OP-Print directly aggregates the privileged teacher-student residual correction along the same rollout; in the black-box setting, it measures restart-based output-basin stability as an observable surrogate for the hidden correction process. Across diverse models and distillation schemes, all baseline attacks remain essentially at random guessing even under gray-box access, whereas OP-Print is broadly validated to be effective in both gray-box and black-box settings.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Inference, Context (archaeology), Tracing, Construct (python library), Residual

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange