Haoran Xue, Nikta Akbarpour, Gias Uddin, Song Wang · 2026 · arXiv
DOI: 10.48550/arXiv.2610.25705 · open-access PDF
Agen pengkodean model bahasa besar (LLM), seperti Claude Code, semakin banyak digunakan untuk audit keamanan, dengan keterampilan khusus tugas yang menyediakan prosedur analisis kerentanan yang dapat diulang. Namun, efektivitas mereka dalam mendeteksi kerentanan nyata dan alasan kegagalan mereka tetap tidak jelas. Kami melakukan studi empiris skala besar pertama dari 37 keterampilan keamanan menggunakan Claude Code dengan empat LLM yang mendasarinya pada 100 kerentanan kernel Linux nyata di sepuluh kelas, total 14800 berjalan.
Machine translation (Meta M2M100); the English original follows.
Large language model (LLM) coding agents, such as Claude Code, are increasingly used for security auditing, with task-specific skills providing reusable vulnerability-analysis procedures. However, their effectiveness in detecting real vulnerabilities and the reasons for their failures remain unclear. We conduct the first large-scale empirical study of 37 security skills using Claude Code with four underlying LLMs on 100 real Linux kernel vulnerabilities across ten classes, totaling 14800 runs. We evaluate vulnerability detection at the function, statement, and vulnerability-class levels. Resul
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.25705, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange