Newest papers

Behavior Beyond Text: Evaluating Refusal Transfer Failure in LLM Agents

Ekagra Gupta, Raj Saha, Aniekan-Abasi Inyang, Adam Al Gharib · 2026 · arXiv

DOI: 10.48550/arXiv.2610.28982 · open-access PDF

Chinese

虽然大型语言模型越来越多地被部署为工具调用代理,但他们的安全评估仍然主要集中在其在文本中拒绝有害请求的能力上。我们调查文本中的拒绝行为是否转移到工具使用中。我们构建了四个领域的2304个提示的基准,包括在文本中,标准工具启用和反对框架的工具启用设置中呈现相同有害或良性的请求的匹配三角形。

Machine translation (Meta M2M100); the English original follows.

Abstract

While large language models are increasingly deployed as tool-calling agents, their safety evaluations still focus primarily on their ability to refuse harmful requests in text. We investigate whether the refusal behavior in text transfers to tool use. We construct a benchmark of 2,304 prompts across four domains, consisting of matched triplets that present the same harmful or benign request in text-only, standard tool-enabled, and adversarially framed tool-enabled settings. We then assess tool-call safety using 20 predefined forbidden actions. Testing five models of varying sizes and families

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange