Newest papers

OP-Print: Tracing Membership Imprints in On-Policy Self-Distillation

Qi Li, Xinchao Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31616 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

OPのオフポリシー蒸留の分布不一致を緩和するために広く採用されています。 効果的な自己蒸留は、学生の独自の出力を使用して、追加の文脈またはデモを監督として強化して、外部教師の必要性をさらに除去します。 この教師フリーの設計は、ポリシーのグレーボックスとブラックボックスの自己蒸留の両方を実用化しますが、そのプライバシーリスクはほとんど未知のままです。 特に、同じトレーニング例の2つの結合された視点を使用して、その記憶を強化することができます。 これによって動機づけられ、我々は、ポリシーの自己蒸留に対するメンバーの結論攻撃の最初の研究を提示します。 我々は、ブラックボックスとブラックボックスの脅威モデル

Machine translation (Meta M2M100); the English original follows.

Abstract

On-policy distillation has been widely adopted for alleviating the distribution mismatch of off-policy distillation. Self-distillation further removes the need for an external teacher by using the student's own outputs, enhanced with extra context or demonstrations, as supervision. This teacher-free design makes on-policy self-distillation practical, yet its privacy risks remain largely unexplored. In particular, using two coupled views of the same training example may strengthen its memorization. Motivated by this, we present the first study of membership inference attacks against on-policy self-distillation. We investigate both gray-box and black-box threat models and propose a unified attack framework, named OP-Print. OP-Print builds on our finding that the supervision imprint in on-policy self-distillation is sparse and structured, concentrating on uncertainty-sensitive correction sites rather than across the entire rollout. Exploiting this property, we construct attack signals tailored to the two access levels: in the gray-box setting, OP-Print directly aggregates the privileged teacher-student residual correction along the same rollout; in the black-box setting, it measures restart-based output-basin stability as an observable surrogate for the hidden correction process. Across diverse models and distillation schemes, all baseline attacks remain essentially at random guessing even under gray-box access, whereas OP-Print is broadly validated to be effective in both gray-box and black-box settings.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Inference, Context (archaeology), Tracing, Construct (python library), Residual

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange