Newest papers

Circuits of LLM Classifiers

Hadi Khalaf, Sajani Vithana, Kunal Talreja, Haewon Jeong, Viveck R. Cadambe, Flavio Calmon · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30076 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

语言模型可以轻松地构建分类器,简单地促使样品进行分类。然而,它们的预测在样品中既不确定性,也不均匀可靠。我们如何将不同模型的预测结合起来以提高可靠性?我们将LLM分类器视为底层目标标签的噪音测量工具。我们的目标是最大限度地提高平均分类准确度,以给定的平均查询成本。为此目的,我们引入了将LLM分类器组织成一个适应性的查询和决策序列的电路。然后,我们描述了最佳电路,获得可实现的精度的基本极限,并提供了实现最佳电路的实际见解。通过七个分类任务,包括安全,健康,政治科学,人力偏好,代理评估,决策和推理,

Machine translation (Meta M2M100); the English original follows.

Abstract

Language models make it easy to build classifiers by simply prompting a model with the sample to classify. However, their predictions are neither deterministic nor uniformly reliable across samples. How can we combine predictions from different models to improve reliability? We view LLM classifiers as noisy measurement instruments of an underlying target label. Our goal is to maximize average classification accuracy at a given average query cost. To this end, we introduce circuits that organize LLM classifiers into an adaptive sequence of queries and decisions. We then characterize the optimal circuit, obtain the fundamental limits of achievable accuracy, and provide practical insights on realizing the optimal circuit. Across seven classification tasks spanning safety, health, political science, human preference, agent evaluation, decision-making and reasoning, our method improves the cost-accuracy frontier over routing and cascading baselines.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Artificial intelligence, Machine learning, Sequence (biology), Sample (material), Random subspace method

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange