Newest papers

BAIT: Can LLMs Find What Biases a Reader? A Benchmark for Cognitive Bias Triggers in News

Yousra Fettach, Tijl De Bie · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28092 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

説得力のある書き込みメカニズムのトリガーは、しばしば読者の信念を形作るために読者の精神的なショートカットを採用し、説得力のある書き込みメカニズムのトリガーを活用し、しばしば読者の気づきなしで信念を形作る。私たちは、そのような心理的なショートカットを活用する言語的機能を評価し、認知的な偏見を引き起こす。大言語モデル(LLMs)が、どのように人々がニュースに遭遇するかをどんどん媒介するように、それをまとめ、ランキングし、それについて質問をするとき、これらの自然な質問は、読者が人間の読者に偏見を与えるテキストの特徴を認識できるかどうかです。私たちは、BAIT(Bias An

Machine translation (Meta M2M100); the English original follows.

Abstract

Persuasive writing exploits cognitive biases, recruiting a reader's mental shortcuts to shape belief, often without the reader noticing. We call the linguistic features exploiting such mental shortcuts cognitive bias triggers. As large language models (LLMs) increasingly mediate how people encounter news, summarizing it, ranking it, and answering questions over it, a natural question is whether they can recognize the features of a text that bias a human reader. We introduce BAIT (Bias Annotation In Text), a resource for detecting cognitive bias triggers in news. BAIT pairs a schema of nine triggers, each specified in an annotator codebook, with 50 English news articles annotated by trained annotators at the article and span levels and adjudicated into a gold standard. On this data we define two tasks, detecting which triggers a text contains and locating the spans that carry them. We then benchmark six frontier LLMs zero-shot and with three or ten in-context examples. The models approach human agreement on which triggers an article contains, but rarely recover its full set of triggers, and they reach less than half of human agreement in locating the passages that carry them. Moreover, they over-predict triggers that come with easy surface cues, and most of the passages they mark for these triggers are not triggers, yet carry the trigger's surface cue far more often than random text. These findings suggest that the evaluated LLMs respond to surface cues rather than to the mechanisms that make a passage manipulative. We release the schema, the codebook, standoff annotations, and the evaluation pipeline.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Cognition, Cognitive bias, Computer science, Ranking (information retrieval), Benchmark (surveying), Set (abstract data type)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange