Newest papers

Investigating the Impact of Learner Errors on Social Bias in Large Language Models

Federico Marcuzzi, Martin Christoph, Iryna Gurevych, Alla Rozovskays · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29896 · cited by 0 works (OpenAlex) · open-access PDF

French

Les critères de référence de préjugés sociaux existants évaluent les grands modèles linguistiques principalement sur l'anglais courant et bien formé, reflétant l'usage des locuteurs natifs et laissant les écrivains non natifs non représentés. Pour résoudre ce fossé, nous enquêtons sur la façon dont les erreurs grammaticales des écrivains non natifs affectent les préjugés sociaux. Nous développons un cadre contrôlé qui injecte les erreurs communes des apprenants dans les prompts et les applique aux trois critères de référence de préjugés couvrant l'alignement stéréotype et la préférence démographique à travers le sexe, la race et la religion. Nous constatons que les erreurs grammaticales augmentent les pré

Machine translation (Meta M2M100); the English original follows.

Abstract

Existing social bias benchmarks evaluate large language models mostly on fluent, well-formed English, reflecting native speaker usage and leaving non-native writers unrepresented. To address this gap, we investigate how grammatical errors by non-native writers affect social bias. We develop a controlled framework that injects common learner errors into prompts and apply it to three bias benchmarks covering stereotype alignment and demographic preference across gender, race, and religion. We find that grammatical errors increase bias, and the effect grows with the error rate. Among the four common error types we consider, preposition errors produce the largest and most consistent increase in stereotype alignment. Our analysis suggests that errors increase bias when they are more likely to interfere with comprehension. Finally, fine-tuning on erroneous texts reduces the bias increase caused by grammatical errors. Our findings show that bias evaluations based exclusively on well-formed prompts may underestimate the bias experienced by users whose English contains grammatical errors.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Stereotype (UML), Affect (linguistics), Psychology, Preference, Gender bias, Computer science

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange