Newest papers

Towards Robust Model Organisms for Eliciting Latent Knowledge

Lennart Finke, Hardik Bhatnagar, Euodia Dodd, Francis Rhys Ward, Yves-Alexandre de Montjoye, Matthias Bethge, Mary Phuong · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26698 · cited by 0 works (OpenAlex) · open-access PDF

French

Un problème clé de l'organisme dans la sécurité de l'IA consiste à induire des connaissances latentes (ELK), extraire des informations d'un modèle qui est mal aligné et plus informé que son superviseur. Par exemple, un modèle mal aligné utilisé pour scanner le code pour les vulnérabilités peut vouloir retenir ses résultats, c'est-à-dire le sac de sable, afin de les exploiter plus tard. Pour étudier l'ELK de manière empirique, nous avons besoin d'organismes de modèle qui reflètent cet environnement. Les organismes de modèle existants tombent en manque à plusieurs égards. Dans ce travail, nous formons les organismes de modèle pour retenir les solutions dans une tâche de détection de vulnérabilité. Nous combinons la distillation de transcription avec la contre

Machine translation (Meta M2M100); the English original follows.

Abstract

A key problem in AI safety is eliciting latent knowledge (ELK), extracting information from a model that is misaligned and more knowledgeable than its overseer. For example, a misaligned model used to scan code for vulnerabilities may want to withhold its findings, i.e., sandbag, in order to exploit them later. To study ELK empirically, we need model organisms that mirror this setting. Existing model organisms fall short in several respects. In this work, we train model organisms to withhold solutions in a vulnerability detection task. We combine transcript distillation with counter-prompting and tamper-resistance training to instill three properties that we argue are critical for a useful ELK testbed: 1) Selectivity, the model sandbags when it thinks the overseer doesn't know the answer and complies otherwise, 2) Robustness, the model resists elicitation, and 3) Retention, the model retains a knowledge advantage over the overseer, as opposed to losing the relevant capability. We train a sequence of model organisms and show that our strongest model substantially outperforms prior models on robustness while keeping the other two properties intact. We hope that our model organisms provide a useful ELK testbed for the community, and that our methodology sparks the development of increasingly challenging model organisms. All our code, models and the dataset are available at https://huggingface.co/collections/lennart-finke/sandbagging.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Exploit, Robustness (evolution), Artificial intelligence, Key (lock), Vulnerability (computing)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange