Newest papers

Predictable performance is suboptimal, and what it means for model routing

Guanhua Zhang, Moritz Hardt · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31428 · cited by 0 works (OpenAlex) · open-access PDF

Japanese

これらの方法は、パフォーマンスの予測に依存します: 同じクエリを用いて、モデルが正しく答えるかどうかを推定します。 ほとんどのルーティング方法は、パフォーマンスの予測が、任務またはパフォーマンスの予測を超えるパフォーマンスの予測が、そのタスクに最適である場合にのみ、パフォーマンスの予測がどれだけ改善できるかを示すことを示しています。 まず、我々は、パフォーマンスの予測が、そのタスクまたはパフォーマンスの予測を超えるパフォーマンスの予測が、候補モデルの損失を改善するために使用できることを示します。 具体的には、我々は、パフォーマンスの予測が、そのパフォーマンスにどれだけ改善できるかを示すために、

Machine translation (Meta M2M100); the English original follows.

Abstract

Large language models (LLMs) differ in capabilities and costs, motivating routing methods that adaptively choose a model for each query. These methods rely on performance prediction: given a query, estimate whether a model will answer it correctly. Most routing methods train separate predictors of candidate models' performance to guide model selection. Our work shows that performance prediction only beats a simple baseline when the candidate model is suboptimal on the task or the performance predictor adds significant capacity relative to the candidate model. Specifically, we take a model's self-confidence, the probability it assigns to its chosen answer, as a baseline and study how much a performance predictor can improve on it. First, we show that any performance predictor that outperforms a model's self-confidence can be used to improve the model's loss: the predictor's advantage transfers exactly in log loss through recalibration. Second, accounting for the predictor's additional parameters, we bound its advantage by the model's suboptimality plus the gain from those parameters. Third, we extend our theoretical results to two-model cascades, which route queries to a more capable model when the first model is predicted to fail. Experiments with 16 language models on four benchmarks show that predictors gain a larger advantage over self-confidence on more suboptimal models. This advantage shrinks as models improve through finetuning. Cascading exhibits the same two patterns: predictor-based routing offers larger gains over confidence-based routing for more suboptimal models, and these gains diminish after finetuning.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Task (project management), Baseline (sea), Routing (electronic design automation), Performance prediction, Performance improvement

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange