Jinyang Li, Denis Emelin, Simone Tedeschi, Weiwei Cheng, Cristina Morariu, Haytham Assem, Jens Lehmann, Emine Yılmaz, Angus Addlesee · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.26511 · cited by 0 works (OpenAlex) · open-access PDF
बड़े भाषा डेटा मॉडल (एलएलएम) पर बनाया गया कॉर्पोरेट एआई सिस्टम को राउटिंग तंत्रों की आवश्यकता होती है जो पैमाने पर प्रतिक्रिया गुणवत्ता और प्रतिक्रिया लाटेनता को संतुलित करते हैं। हालांकि, मौजूदा एलएलएम राउटिंग बेंचमार्क सभी उम्मीदवार मॉडल परिणामों की पूर्ण निरीक्षणशीलता का अनुमान लगाते हैं, एक प्रथम श्रेणी के मीट्रिक के रूप में लाटेनशीलता को कम जोर देते हैं, और वास्तविक दुनिया के सेटिंग्स के साथ अनियमित heterogeneous प्रोटोकॉलों के बीच टुकड़े मूल्यांकन। हम RouterWorld, वार्तात्मक GenAI में LLM राउटिंग के लिए एक व्यापक बेंचमार्क को पेश करते हैं। महत्वपूर्ण बात यह है कि RouterWorld 371k
Machine translation (Meta M2M100); the English original follows.
Conversational AI systems built on Large Language Models (LLMs) require routing mechanisms that balance answer quality and response latency at scale. However, existing LLM routing benchmarks assume full observability of all candidate model outcomes, under-emphasize latency as a first-class metric, and fragment evaluation across heterogeneous protocols misaligned with real-world settings. We introduce RouterWorld, a comprehensive benchmark for LLM routing in conversational GenAI. RouterWorld unifies 371k single-turn responses and 76k multi-turn agentic trajectories, adopts natural user interactions with a standardized LLM-as-a-judge protocol, and decouples efficiency from transient pricing via hardware-grounded latency annotations. Crucially, RouterWorld formalizes routing under partial observability through two setups: Cold-Start, which restricts learning to sparse, static logs, and Voyager, which models continuous data streams where routing decisions shape future training distributions. Evaluating 13 routing baselines, we find that most routers over-emphasize accuracy while ignoring latency under partial observability. Code and benchmark data are publicly available.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Observability, Latency (audio), Static routing, Benchmark (surveying), Link-state routing protocol
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.26511, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange