Newest papers

Differentially Private Sparse Reward Estimation with Preference Feedback

Meng Ding, Mingxi Lei, Jie Zhang, Jinyan Liu, Di Wang · 2026 · arXiv

DOI: 10.48550/arXiv.2610.30099 · open-access PDF

Spanish

Reinforcement Learning with Human Feedback (RLHF) se ha convertido en un paradigma central para alinear grandes modelos de lenguaje con valores humanos. Sin embargo, la dependencia de datos de preferencia sensibles requiere estrictas garantías de privacidad diferencial (DP) para prevenir la fuga de información del usuario. Los métodos de alineación privada existentes suelen asumir parámetros de recompensa densos, resultando en límites de error de estimación que se escalan polinomialmente con el ambiente {tamaño del modelo $d$}. Tal dependencia es prohibida en modelos modernos {overparameterized} donde $d$ es grande. En este trabajo, estudiamos \hemp{Private Sparse RLHF} pr

Machine translation (Meta M2M100); the English original follows.

Abstract

Reinforcement Learning with Human Feedback (RLHF) has become a central paradigm for aligning Large Language Models with human values. However, the reliance on sensitive preference data necessitates rigorous differential privacy (DP) guarantees to prevent user information leakage. Existing private alignment methods typically assume dense reward parameterizations, resulting in estimation error bounds that scale polynomially with the ambient {model size $d$}. Such dependence is prohibitive in modern {overparameterized models} where $d$ is vast. In this work, we study \emph{Private Sparse RLHF} pr

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange