Oleg Grynets, Vasyl Lyashkevych, Andrii Salyk, Dmytro Koziuberda, Dmytro Kostetskyi, Oleh Kaskun, Olena Pochernina, Alona Seletska, Oleksii Ilchuk, Viktoriia Tsymbal, Daryna Tukalo, Bohdan Saukh, Ivan V. Fedorchuk, Hryhorii Petrenko · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.25659 · cited by 0 works (OpenAlex) · open-access PDF
该论文介绍了一种方法,用于应用程序特定的能力配置LLM和其实现作为一个集装箱化测试床。该方法将每个应用程序视为一个扩展单位:Oracle-to-PostgreSQL迁移代理人、基于文档的聊天机器人或企业支持的自动化代理人运行在自己的容器后面一个小合同,带来了自己的数据集,评估器和拥有属性,并与其仪表板一起登陆。测量积累在每个模型的分散版本中。这些方法将测试中的模型与审判员和代理人分开,冻结实验和哈希计划,拒绝比较来自不同背景的值,演示仪器之前的诊断,并将试验器应用于零检查器,并且不会对独立的应用程序进行迁移,尽管模型的选择并非由自动化
Machine translation (Meta M2M100); the English original follows.
Public leaderboards rank large language models (LLMs) in general, but an engineer has to choose a model for a specific application, architecture and cost. This paper presents a method for application-specific capability profiling of LLMs and its realization as a containerized testbed. The method treats each application as a unit of extension: an Oracle-to-PostgreSQL migration agent, a documentation-grounded chatbot or an enterprise-support automation agent runs in its own container behind a small contract, brings its own dataset, evaluator and owned attributes, and is onboarded together with its dashboards. Measurements accumulate in a versioned, sparse metamodel of every model. The method separates the model under test from judges and agents, freezes and hashes plans, refuses to compare values from different contexts, diagnoses the instrument before the system, and applies a null-agent check and a noise rule whose operating characteristics are quantified by simulation. In the evaluated automation conditions, model choices produced larger differences on selected observable outcomes than framework choices, although the experimental design does not establish a general causal ranking of these factors. The experiments also show that instruction tuning is deployment-specific and that system-reported success can exceed independently observed success: of 216 migration units checked against a real Oracle database, 108 were demonstrably correct. These measurements establish application-specific observations, not a universal model ranking.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Automation, Oracle, Profiling (computer programming), Metamodeling, Reuse
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.25659, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange