Newest papers

Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering

Kuniko Paxton, Ifeoluwa Wuraola, Dhavalkumar Thakker · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29657 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

在资源限制设置中部署小型和量化语言模型提出了一个重要的公平性问题:当压缩模型产生更少偏见的响应时,是否反映更公平的行为或更大的弃权?我们在问答(QA)中使用BBQ基准在多个模型大小和量化设置中研究这个问题,但从Qwen3,Gemma-3和SmolLM2开始,偏见没有显示简单的扩展趋势和量化对不同配置产生影响。我们还研究了观察到的趋势是否在替代提示式中保持稳定,以及当“未知”选项在强迫选择设置中被删除时。我们的结果显示,较大的模型实现了更高的QA准确性,但偏见没有显示简单的扩展趋势和量化对不同配置产生影响。值得注意的是,在26个配置中,

Machine translation (Meta M2M100); the English original follows.

Abstract

The deployment of small and quantized language models in resource-constrained settings raises an important fairness question: when a compressed model produces fewer biased responses, does this reflect fairer behavior or greater abstention? We study this problem in Question Answering (QA) using the BBQ benchmark across multiple model sizes and quantization settings from Qwen3, Gemma-3, and SmolLM2. We also examine whether the observed trends remain stable across alternative prompt formulations and when the ``Unknown'' option is removed in a forced-choice setting. Our results show that larger models achieved higher QA accuracy, but bias showed no simple scaling trend and quantization effects varied across configurations. Notably, 20 of the 26 configurations with fewer biased responses also show increased over-abstention, indicating that lower observed bias can partly reflect greater abstention. These findings show that lower observed bias can partly reflect greater abstention, and that fairness evaluation should consider biased responses, abstention, prompt formulation, and answer-option design jointly.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Quantization (signal processing), Scaling, Question answering, Econometrics, Computer science, Psychology

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange