Newest papers

Towards Robust Model Organisms for Eliciting Latent Knowledge

Lennart Finke, Hardik Bhatnagar, Euodia Dodd, Francis Rhys Ward, Yves-Alexandre de Montjoye, Matthias Bethge, Mary Phuong · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26698 · cited by 0 works (OpenAlex) · open-access PDF

Korean

인공지능 보안의 핵심 유기체 문제는 잠재적 인 지식 (ELK)을 유발하는 것이며, 모형이 잘못 조정되고 감독자보다 더 지식이 풍부한 모델에서 정보를 추출합니다. 예를 들어, 취약점에 대한 코드를 스캔하는 데 사용되는 잘못 조정 된 모델은 발견, 즉 샌드백을 저장하고 나중에 이용하기 위해 샌드백을 원할 수 있습니다. ELK를 경험적으로 연구하기 위해 우리는이 설정을 반영하는 모델 유기체가 필요합니다. 기존 모델 유기체는 여러 가지 측면에서 부족합니다. 이 작업에서 우리는 취약점 탐지 작업에서 솔루션을 저장하기 위해 모델 유기체를 훈련합니다. 우리는 트랜스프리트 디스플레이션과 대변 펌프팅 및 방패 저

Machine translation (Meta M2M100); the English original follows.

Abstract

A key problem in AI safety is eliciting latent knowledge (ELK), extracting information from a model that is misaligned and more knowledgeable than its overseer. For example, a misaligned model used to scan code for vulnerabilities may want to withhold its findings, i.e., sandbag, in order to exploit them later. To study ELK empirically, we need model organisms that mirror this setting. Existing model organisms fall short in several respects. In this work, we train model organisms to withhold solutions in a vulnerability detection task. We combine transcript distillation with counter-prompting and tamper-resistance training to instill three properties that we argue are critical for a useful ELK testbed: 1) Selectivity, the model sandbags when it thinks the overseer doesn't know the answer and complies otherwise, 2) Robustness, the model resists elicitation, and 3) Retention, the model retains a knowledge advantage over the overseer, as opposed to losing the relevant capability. We train a sequence of model organisms and show that our strongest model substantially outperforms prior models on robustness while keeping the other two properties intact. We hope that our model organisms provide a useful ELK testbed for the community, and that our methodology sparks the development of increasingly challenging model organisms. All our code, models and the dataset are available at https://huggingface.co/collections/lennart-finke/sandbagging.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Exploit, Robustness (evolution), Artificial intelligence, Key (lock), Vulnerability (computing)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange