Newest papers

TIDE: Tree-Induced Fine-Grained Diagnostic Evaluation from Model Behaviors

Jiayi Chen, Hui Li, Yujiong Shen, Qiyuan Peng, Yanxin Li, Chenhao Huang, Shaofan Liu, Guoqiang Zhang, Junjie Ye, Shihan Dou, Ming Zhang, Maxm Pan, Tao Gui, Qi Zhang, Xuanjing Huang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29985 · cited by 0 works (OpenAlex) · open-access PDF

Arabic

وتتضمن هذه النتائج معلومات محدودة حول القدرات الأساسية للمنتج، ولكن فهم هذه القدرات أمر حاسم لتطوير النماذج وتقييمها. في هذا الإطار، ونحن نقدم تقييمًا متكاملًا واحدًا، مثل دقة، معدل المراجعة، أو معدل القسم. هذه النتائج المتكاملة توفر معلومات محدودة عن القدرات الأساسية للمنتج، ومع ذلك، فإن فهم هذه القدرات أمر حاسم لتطوير النماذج وتقييم النماذج: في كل نقطة، ونحن نقترح تقييمًا متكاملًا (TIDE) الذي يقوم بتقييم التشخيص كالتقييم النماذج إلى مجموعة وتقييم المعلومات المتعلقة بالممارسة بينهم. ونحن نستخدم TIDE لعدد من النماذج الأ

Machine translation (Meta M2M100); the English original follows.

Abstract

Benchmark performance is the dominant standard for comparing large language models (LLMs), typically summarized by a single aggregate score, such as accuracy, pass rate, or rubric score. This aggregate score provides limited information about a model's underlying capabilities, yet understanding these capabilities is crucial for model development and evaluation. To this end, we propose Tree-Induced Diagnostic Evaluation (TIDE), which recasts diagnostic evaluation as behavior-driven decision-tree induction to perform fine-grained capability diagnosis. TIDE recursively builds a hierarchical capability tree from benchmark samples and model scores: at each node, an LLM judge proposes candidate splits and performs sample-to-group classification, and behavior-based information gain selects among them. We apply TIDE to five models across four benchmarks to demonstrate its effectiveness, and find fine-grained capability contrasts invisible at the aggregate level: Claude Opus 4.6 achieves 100% on microeconomic market reasoning where GPT-5.4 scores 83.3%, but trails GPT-5.4 on contract law reasoning with 77.8% versus 88.9%. We hope TIDE can be extended to past and future benchmarks so that the rich behavioral signal in evaluation results is fully leveraged for understanding model capabilities.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Benchmark (surveying), Rubric, Computer science, Aggregate (composite), Machine learning, Artificial intelligence

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange