Newest papers

How Effectively Do Security Skills Guide Coding Agents in Vulnerability Detection in Linux Kernel?

Haoran Xue, Nikta Akbarpour, Gias Uddin, Song Wang · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.25705 · cited by 0 works (OpenAlex) · open-access PDF

Korean

클라우드 코드와 같은 대형 잠재적 취약성 모델 (LLM) 코딩 에이전트는 보안 감사를 위해 점점 더 많이 사용되고 있으며, 작업 특정 기술로 재사용 가능한 취약성 분석 절차를 제공합니다. 그러나, 실제 취약성 및 실패의 이유를 감지하는 데 그들의 효율성은 불분명합니다. 우리는 클라우드 코드를 사용하여 37 개의 보안 기술을 사용하여 4 개의 기본적인 LLM을 사용하여 10 개의 클래스에 걸쳐 100 개의 실제 리눅스 코어 취약성을 정확하게 분류합니다. 우리는 기능, 진술 및 취약성 클래스 수준에서 취약성 감지에 대한 한계를 평가합니다. 결과는 실행자의 24.4 퍼센트가 올바른 기능을 찾고, 18.3 퍼센트가 기능과 보안 진술

Machine translation (Meta M2M100); the English original follows.

Abstract

Large language model (LLM) coding agents, such as Claude Code, are increasingly used for security auditing, with task-specific skills providing reusable vulnerability-analysis procedures. However, their effectiveness in detecting real vulnerabilities and the reasons for their failures remain unclear. We conduct the first large-scale empirical study of 37 security skills using Claude Code with four underlying LLMs on 100 real Linux kernel vulnerabilities across ten classes, totaling 14800 runs. We evaluate vulnerability detection at the function, statement, and vulnerability-class levels. Results show that 24.4 percent of runs locate the correct function, 18.3 percent identify both the function and responsible statement, and 15.7 percent correctly classify the vulnerability. Our failure analysis reveals limitations in evidence acquisition, state-relation reasoning, code interpretation, candidate verification, investigation coverage, and reporting. Based on these findings, we propose SKILL-SIF, a self-improving framework that iteratively evolves security skills by learning from missed vulnerabilities and failure patterns. We evolve five top-performing skills using the original benchmark and evaluate them on 100 previously unseen vulnerabilities. SKILL-SIF improves fully correct vulnerability detection by up to eight times (from 2 percent to 16 percent) on unseen vulnerabilities, while aggregate accuracy across four LLMs increases by 1.85 times (from 6.05 percent to 11.20 percent). These findings highlight the limitations of existing security skills and demonstrate the potential of failure-driven skill evolution to improve vulnerability detection in coding agents.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Vulnerability (computing), Coding (social sciences), Benchmark (surveying), Computer security, Secure coding

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange