Newest papers

Benchmarking Coding Agents for System-Level Performance Optimization

Yanze Zhang, Maggie Yongqi Guan, Shudong Liu, Qi Hao, Kanye Ye Wang, Meng Qu, Xiaobo Zhou · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29695 · cited by 0 works (OpenAlex) · open-access PDF

Arabic

يبدو أن المساعدات الرقمية أصبحت نموذجاً جديداً للتحكم في مهام الهندسة البرمجية. ومع ذلك، تتطلب المساعدات الرقمية الحالية أساساً تقييم المساعدات على مستوى المكتب أو المكتب، مما يترتب عليه من إمكاناتها في أنظمة البرمجيات الموزعة أكثر تعقيداً. في الممارسة، فإن العديد من التطبيقات الحديثة تطبق بنماذج ميكروسوفت، حيث تصبح نموذجاً جديداً للتحكم من خلال التفاعلات بين العديد من الخدمات المستخدمة بشكل مستقل، مما يطلب المساعدات الرقمية إعادة تقييمها على المستوى المحلي من أجل التغيير. لتطوير هذه القدرة، نحن نقدم MicroPerf-Bench، نموذجاً جديداً لتقييم الم

Machine translation (Meta M2M100); the English original follows.

Abstract

Coding agents are becoming an increasingly important paradigm for automating software engineering tasks. However, existing coding benchmarks primarily evaluate agents at the function or repository level, leaving their capabilities in more complex distributed software systems largely unexplored. In practice, many modern applications adopt microservice architectures, where end-to-end performance emerges from interactions among multiple independently running services, requiring agents to reason beyond local code changes. To study this capability, we introduce MicroPerf-Bench, a benchmark for evaluating coding agents on end-to-end performance optimization in microservice systems. Constructing such a benchmark is challenging because reproducible system-level performance fixes are sparse and difficult to attribute. We address this challenge by injecting empirically grounded performance anti-patterns into real-world open-source systems while preserving functionality and ensuring measurable performance degradation. The resulting benchmark contains 65 optimization instances across seven systems and nine anti-patterns, each with an executable environment, functional tests, and mixed-workload evaluation. Across 1,755 runs covering nine agent-model configurations, the best configuration recovers only 42.0% of the injected performance degradation. The dominant failure is not invalid code, but functionally correct and constraint-compliant changes that fail to improve end-to-end performance, highlighting system-level bottleneck localization as a key challenge for current coding agents. The source code and data are available at https://github.com/zyzcjnb/microperf.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Benchmarking, Computer science, Bottleneck, Executable, Benchmark (surveying), Coding (social sciences)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange