Newest papers

A Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM Era

Eric Liang, Alan Luo, Kaiwen Yuan · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29196 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

最近的VLM在Vision-Language数据模型中发现了脆弱性,而VLM在语言提示和生成解码中发现了脆弱性,显示出极大地提高了零射击和开放式词汇检测能力。结果是,VLM在安全危险的现实世界系统中迅速部署在边缘:但是,边缘VLM的物理对抗模式的强度没有得到系统研究,并且与现代对象检测器的输出差异在根本上造成了公平评估的方法上的挑战。为了直接比较这两个家庭,我们在一个结果上评估每个选择的代表模型:是否在人体上使用的对抗模型检测到模型失败。我们通过采用统一的测量,条件攻击成功率(cASR)来引入这一报告,我们通过家庭的测

Machine translation (Meta M2M100); the English original follows.

Abstract

Recent advancements in Vision-Language Models (VLMs) enable end-to-end visual grounding via language prompts and generative decoding, demonstrating largely improved zero-shot and open-vocabulary detection capabilities. As a result, VLMs are rapidly being deployed at the edge in safety-critical real-world systems. However, the physical adversarial robustness of edge VLMs has not been systematically studied, and the fundamental differences in output with modern object detectors create methodological challenges for fair evaluation. To compare the two families directly, we evaluate every selected representative model on one outcome: whether an adversarial patch worn on a person evades detection of the model. We report this by introducing a unified metric, conditional attack success rate (cASR), counted only on images the model still localizes under a matched gray occluder, and partition every failure into omission (no box), substitution (a box on the patch), and displacement (a box on neither). We benchmark the physical adversarial vulnerability of compact grounding VLMs against a range of object detectors, and find that the VLMs can be as vulnerable as the detectors. We disclose that failure mode is largely associated with the output format of a model rather than its architecture family. Of the three, only omission is visible to a downstream system; displacement and substitution return confident, well-formed boxes, making them potentially more dangerous failures. Further, we find that, under distribution shift, VLMs that primarily failed by omission instead shift to failing more by displacement/substitution. Thus, a grounding VLM can inherit the detector's vulnerability while showing the possibility to adopt more dangerous failure modes. Our code and data are available at https://github.com/ericliang1/physical-adversarial-robustness.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Adversarial system, Robustness (evolution), Computer science, Artificial intelligence, Computer security, Vulnerability (computing)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange