Newest papers

BAIT: Can LLMs Find What Biases a Reader? A Benchmark for Cognitive Bias Triggers in News

Yousra Fettach, Tijl De Bie · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28092 · cited by 0 works (OpenAlex) · open-access PDF

Korean

설득력있는 글쓰기 트리거는 지능적 편견을 이용하여 독자의 정신적 단축을 믿음을 형성하기 위해 종종 독자가 알지 못하는 경우를 모집합니다. 우리는 그러한 정신적 단축을 활용하는 언어적 특성을 추출하는 지능적 편견을 이용합니다. 대형 언어 모델 (LLM)이 뉴스에서 지능적 편견을 탐지하기위한 리소스 인 BAIT (Bias Annotation In Text)를 소개합니다. 대형 언어 모델 (LLMs)은 뉴스에서 지능적 편견을 탐지하기위한 리소스를 점점 더 많이 중시합니다. BAIT는 뉴스에서 뉴스를 만나는 방법을 요약하고, 그것을 정렬하고, 그것에 대한 항목 코드 책에 지정하고, 독자 코드 책에 지정하는 9 개의 계획

Machine translation (Meta M2M100); the English original follows.

Abstract

Persuasive writing exploits cognitive biases, recruiting a reader's mental shortcuts to shape belief, often without the reader noticing. We call the linguistic features exploiting such mental shortcuts cognitive bias triggers. As large language models (LLMs) increasingly mediate how people encounter news, summarizing it, ranking it, and answering questions over it, a natural question is whether they can recognize the features of a text that bias a human reader. We introduce BAIT (Bias Annotation In Text), a resource for detecting cognitive bias triggers in news. BAIT pairs a schema of nine triggers, each specified in an annotator codebook, with 50 English news articles annotated by trained annotators at the article and span levels and adjudicated into a gold standard. On this data we define two tasks, detecting which triggers a text contains and locating the spans that carry them. We then benchmark six frontier LLMs zero-shot and with three or ten in-context examples. The models approach human agreement on which triggers an article contains, but rarely recover its full set of triggers, and they reach less than half of human agreement in locating the passages that carry them. Moreover, they over-predict triggers that come with easy surface cues, and most of the passages they mark for these triggers are not triggers, yet carry the trigger's surface cue far more often than random text. These findings suggest that the evaluated LLMs respond to surface cues rather than to the mechanisms that make a passage manipulative. We release the schema, the codebook, standoff annotations, and the evaluation pipeline.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Cognition, Cognitive bias, Computer science, Ranking (information retrieval), Benchmark (surveying), Set (abstract data type)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange