Newest papers

Effects of Model Scale and Quantization on Social Bias and Abstention in Question Answering

Kuniko Paxton, Ifeoluwa Wuraola, Dhavalkumar Thakker · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29657 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Mengimplementasikan model bahasa kecil dan kuantizasi dalam pengaturan sumber daya membangkitkan pertanyaan keadilan yang penting: ketika model yang dikompresi menghasilkan jawaban yang kurang bias, apakah ini mencerminkan perilaku yang lebih adil atau abstensi yang lebih besar? Kami mempelajari masalah ini dalam Answering Question (QA) menggunakan referensi BBQ di berbagai ukuran model dan pengaturan kuantizasi dari Qwen3, Gemma-3, dan SmolLM2. Kami juga memeriksa apakah tren yang diamati tetap stabil di berbagai formulasi prompt alternatif dan ketika opsi "Unknown" dihapus dalam pengaturan pilihan paksa. Hasil kami menunjukkan bahwa model yang lebih besar mencapai akurasi QA yang lebih tinggi, tetapi bias tidak menunjukkan tren skala sederhana dan kuantizasi yang mempengaruhi variasi. Terutama, 20 dari 26 konfigurasi dengan jawaban bias yang lebih sedikit juga menunjukkan

Machine translation (Meta M2M100); the English original follows.

Abstract

The deployment of small and quantized language models in resource-constrained settings raises an important fairness question: when a compressed model produces fewer biased responses, does this reflect fairer behavior or greater abstention? We study this problem in Question Answering (QA) using the BBQ benchmark across multiple model sizes and quantization settings from Qwen3, Gemma-3, and SmolLM2. We also examine whether the observed trends remain stable across alternative prompt formulations and when the ``Unknown'' option is removed in a forced-choice setting. Our results show that larger models achieved higher QA accuracy, but bias showed no simple scaling trend and quantization effects varied across configurations. Notably, 20 of the 26 configurations with fewer biased responses also show increased over-abstention, indicating that lower observed bias can partly reflect greater abstention. These findings show that lower observed bias can partly reflect greater abstention, and that fairness evaluation should consider biased responses, abstention, prompt formulation, and answer-option design jointly.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Quantization (signal processing), Scaling, Question answering, Econometrics, Computer science, Psychology

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange