Newest papers

Rethinking Post-hoc Debiasing of Large Pretrained Models

Divyaansh Mertia, Chirag Bhutra, Himanshu Shekhar, Shubham Bagwari, Pratik Mazumder · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29688 · cited by 0 works (OpenAlex) · open-access PDF

German

Große prätrainierte Modelle wie CLIP werden weit verbreitet als gefrorene Feature-Extraktoren eingesetzt. Allerdings erben ihre Embeddings gefälschte Korrelationen von Trainingsdaten, die Bias in die Features einführen. Eine wachsende Gruppe von Forschungen hat versucht, dieses Problem durch Post-Hoc-Debiasing anzugehen, wobei der Rückenmark unberührt bleibt. Wir entdecken, dass trotz der unterschiedlichen Mechanismen diese Methoden die Bias-Funktion erfordern, um in Text identifiziert und ausgedrückt zu werden. Diese Abhängigkeit ist nicht harmlos. In dieser Arbeit zeigen wir experimentell, dass diese Anforderung in bestimmten Szenarien eine unvorhergesehene Strafe erleidet. Zum Beispiel werden auf dem UrbanCars-Ben

Machine translation (Meta M2M100); the English original follows.

Abstract

Large pretrained models such as CLIP are widely used as frozen feature extractors. However, their embeddings inherit spurious correlations from training data, introducing bias into the features. A growing body of research has sought to address this issue through post hoc debiasing, leaving the backbone untouched. We discover that despite differing mechanisms, these methods require the bias feature to be identified and expressed in text. This dependency is not innocuous. In this work, we experimentally demonstrate that this requirement incurs an unforeseen penalty in certain scenarios. For example, on the UrbanCars benchmark, having two co-occurring bias attributes, the recent methods underperform a vanilla classifier on biased frozenbackbone features, and even providing ground-truth bias identities does not close the gap. We show that the pattern that such methods seek is readily observable in the frozen geometry. Specifically, cross-class training pairs placed closest together by the frozen encoder almost always share a spurious attribute, regardless of the bias type, without requiring access to the bias labels. Based on this, we propose a novel approach to identify such pairs and learn a single linear map that transforms the features to perform post-hoc debiasing. The proposed approach achieves competitive performance across all datasets without access to bias attribute annotations or any LLMs.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Debiasing, Spurious relationship, Computer science, Classifier (UML), Artificial intelligence, Feature (linguistics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange