Newest papers

AgentPhishing: A Benchmark of LLM Email Agents Under Adaptive Attack

Justin Tan, Miao Li, Ding Zhao · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31404 · cited by 0 works (OpenAlex) · open-access PDF

Arabic

وبالتالي، يخلق الفيسبوك مخاطر أمنية وأمنية تعتمد فقط على ما إذا كان الممثلون يعترفون بالرسائل الخبيثة، ولكن أيضاً على كيفية التعامل معها. تم تطوير مجموعات بيانات الفيسبوك الحالية أساساً لتحديد الاتجاهات التقليدية، مما يجعله الممثلون قادرين على التعامل مع الطلبات الخبيثة بشكل غير كاف على حد سواء. لتوضيح مخاطر الخصوصية والسلامة التي قد تنقصها الممثلين الحاليين، ونحن نقدم AgentPhishing، علامة قياسية لتقييم ما إذا كان الهجمات الخبيثة يمكن أن تمنع التعرف على هذه الهجمات الخبيثة وتجعل الممثلين عبر البريد الإلكتروني يتوافقون مع الطلبات الخبيثة. ونحن نقدم Phish

Machine translation (Meta M2M100); the English original follows.

Abstract

Computer-use agents increasingly handle private information and act on users' behalf across desktop and web applications, including email. Phishing therefore poses privacy and security risks that depend not only on whether agents recognize malicious messages, but also on how they respond. Existing phishing datasets were developed primarily for conventional detection, leaving agent susceptibility to persuasive malicious requests insufficiently characterized. To expose vulnerabilities that pre-existing datasets may miss, we introduce AgentPhishing, a benchmark for evaluating whether persuasive phishing attacks can evade detection and induce LLM email agents to comply with malicious requests. We develop PhishOpt, which optimizes reusable email-generation skills against individual agents under fixed benchmark constraints and a realism constraint. We analyze the generated emails using Cialdini's persuasion principles to identify strategies associated with agent failures and use these findings to construct a universal generation skill. Our evaluation shows that agents can handle pre-existing phishing emails safely yet remain susceptible to attacks adapted to agent behavior, while in an AgentPhishing evaluation of nine agents, four exhibit phishing-detection rates below $10\%$ and malicious-request compliance rates above $45\%$. These results highlight the limits of inferring agent safety from conventional phishing evaluations and the need to assess how agents respond to persuasive malicious requests.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Phishing, Computer science, Persuasion, Benchmark (surveying), Computer security, Construct (python library)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange