Beyond Leaderboards: A Method for Application-Specific Capability Profiling Больших Моделей Языка / Beyond Leaderboards: A Method for Application-Specific Capability Profiling of Large Language Models (2026) (Russian)

Oleg Grynets, Vasyl Lyashkevych, Andrii Salyk, Dmytro Koziuberda, Dmytro Kostetskyi, Oleh Kaskun, Olena Pochernina, Alona Seletska · 2026 · arXiv

Russian

Этот документ представляет метод для профилирования возможностей, специфических для конкретного приложения, LLM и его реализации в качестве контейнерного теста. Метод рассматривает каждое приложение как единицу расширения: миграционный агент Oracle-to-PostgreSQL, чат-бот на основе документации или агент автоматизации, поддерживающий предприятие, работает в собственном контейнере за небольшим контрактом, приносит свой собственный набор данных, оценщик и принадлежащие ему атрибуты, и на борту вместе с i

Abstract

Public leaderboards rank large language models (LLMs) in general, but an engineer has to choose a model for a specific application, architecture and cost. This paper presents a method for application-specific capability profiling of LLMs and its realization as a containerized testbed. The method treats each application as a unit of extension: an Oracle-to-PostgreSQL migration agent, a documentation-grounded chatbot or an enterprise-support automation agent runs in its own container behind a small contract, brings its own dataset, evaluator and owned attributes, and is onboarded together with i

Is this paper real? This page was built from OpenAlex and arXiv records, not generated. Software can verify any reference in one call: GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.25659, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txt