Newest papers

The Development of Sycophancy in Language Models: Tracing Vulnerabilities from Pretraining to Post-Training

Haonan Huang, Chengkun Cai · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26655 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Model bahasa dapat mengubah jawaban yang berkomitmen untuk menampung pendapat pengguna atau pushback tanpa bukti baru. Namun, ketika perilaku ini berkembang selama pelatihan dan apa yang diwariskan oleh model yang dirilis dari tahap sebelumnya tetap tidak jelas. Kami mengikuti 741 titik kontrol dari 30 garis model terbuka, dari pra-latihan awal hingga model yang dirilis. Kami menanam putusan awal pada skenario moral dan mengukur seberapa jauh sebuah model mengikuti pendapat pengguna yang dinyatakan atau meninggalkan putusannya di bawah tekanan. Dalam sebagian besar garis perilaku hadir sebelum pasca pelatihan, dan kekuatan rata-rata berubah di kedua arah, pada tahap yang berbeda di seluruh garis. Sebuah pola yang lebih jelas muncul di tingkat pertanyaan individu. Di luar jawaban moral tanpa tekanan, perilaku pra-latihan memprediksi sedikit dari pertanyaan mana model yang dirilis menghasilkan. bahwa pola terbentuk selama pasca-pelatihan: pada rute dengan pengawasan

Machine translation (Meta M2M100); the English original follows.

Abstract

Language models can change a committed answer to accommodate a user's opinion or pushback without new evidence. However, when this behaviour develops during training and what a released model inherits from earlier stages remain unclear. We follow 741 checkpoints of 30 open model lineages, from early pretraining to released models. We plant initial verdicts on moral scenarios and measure how far a model follows a user's stated opinion or abandons its verdict under pressure. In most lineages the behaviour is present before post-training, and its average strength changes in either direction, at different stages across lineages. A clearer pattern emerges at the level of individual questions. Beyond responses without pressure, pretraining behaviour predicts little of which questions a released model yields on. That pattern forms during post-training: on routes with supervised fine-tuning, the fine-tuning checkpoint predicts part of it and the preference-optimization checkpoint most of it. A sibling trained from the same base along another route predicts far less: for Tülu-3 8B, its own preference-stage checkpoint removes 85% of the remaining prediction error, a sibling 19%. Internal states add predictive information without changing this ranking. Similar average scores conceal different vulnerabilities, and training history matters for auditing a released model.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Measure (data warehouse), Psychology, Audit, Artificial intelligence, Cognitive psychology

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange