Newest papers

BAIT: Can LLMs Find What Biases a Reader? A Benchmark for Cognitive Bias Triggers in News

Yousra Fettach, Tijl De Bie · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28092 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

Os mecanismos de avaliação de texto persuasivos exploram os triggers cognitivos, recrutando os atalhos mentais de um leitor para moldar a crença, muitas vezes sem que o leitor perceba. Nós chamamos as características linguísticas que exploram esses atalhos mentais de triggers de bias cognitivos. À medida que os grandes modelos de linguagem (LLMs) mediam cada vez mais como as pessoas se deparam com as notícias, resumindo-as, classificando-as e respondendo a perguntas sobre elas, uma pergunta natural é se eles podem reconhecer as características de um texto que bias um leitor humano. Introduzimos o BAIT (Bias Annotation In Text), um recurso para detectar os bias cognitivos em notícias. O BAIT combina um esquema de nove triggers, cada um especificado em

Machine translation (Meta M2M100); the English original follows.

Abstract

Persuasive writing exploits cognitive biases, recruiting a reader's mental shortcuts to shape belief, often without the reader noticing. We call the linguistic features exploiting such mental shortcuts cognitive bias triggers. As large language models (LLMs) increasingly mediate how people encounter news, summarizing it, ranking it, and answering questions over it, a natural question is whether they can recognize the features of a text that bias a human reader. We introduce BAIT (Bias Annotation In Text), a resource for detecting cognitive bias triggers in news. BAIT pairs a schema of nine triggers, each specified in an annotator codebook, with 50 English news articles annotated by trained annotators at the article and span levels and adjudicated into a gold standard. On this data we define two tasks, detecting which triggers a text contains and locating the spans that carry them. We then benchmark six frontier LLMs zero-shot and with three or ten in-context examples. The models approach human agreement on which triggers an article contains, but rarely recover its full set of triggers, and they reach less than half of human agreement in locating the passages that carry them. Moreover, they over-predict triggers that come with easy surface cues, and most of the passages they mark for these triggers are not triggers, yet carry the trigger's surface cue far more often than random text. These findings suggest that the evaluated LLMs respond to surface cues rather than to the mechanisms that make a passage manipulative. We release the schema, the codebook, standoff annotations, and the evaluation pipeline.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Cognition, Cognitive bias, Computer science, Ranking (information retrieval), Benchmark (surveying), Set (abstract data type)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange