Newest papers

A Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM Era

Eric Liang, Alan Luo, Kaiwen Yuan · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29196 · cited by 0 works (OpenAlex) · open-access PDF

French

Les avancées récentes du système de détection des vulnérabilités de VLM sont plutôt des avancées potentialisées dans les modèles de données de vision-langue (VLMs) permettant une mise à la terre visuelle de bout en bout via les prompts linguistiques et le décodage génératif, démontrant en grande partie des capacités de détection de zéro coup et de vocabulaire ouvert améliorées. En conséquence, les VLMs sont rapidement déployés à l'avant-garde dans les systèmes du monde réel dangereux et dangereux. Cependant, la robustesse physique de l'adversaire utilisé sur le sol des VLMs de bord n'a pas été systématiquement étudiée, et les différences fondamentales de sortie avec nos détecteurs d

Machine translation (Meta M2M100); the English original follows.

Abstract

Recent advancements in Vision-Language Models (VLMs) enable end-to-end visual grounding via language prompts and generative decoding, demonstrating largely improved zero-shot and open-vocabulary detection capabilities. As a result, VLMs are rapidly being deployed at the edge in safety-critical real-world systems. However, the physical adversarial robustness of edge VLMs has not been systematically studied, and the fundamental differences in output with modern object detectors create methodological challenges for fair evaluation. To compare the two families directly, we evaluate every selected representative model on one outcome: whether an adversarial patch worn on a person evades detection of the model. We report this by introducing a unified metric, conditional attack success rate (cASR), counted only on images the model still localizes under a matched gray occluder, and partition every failure into omission (no box), substitution (a box on the patch), and displacement (a box on neither). We benchmark the physical adversarial vulnerability of compact grounding VLMs against a range of object detectors, and find that the VLMs can be as vulnerable as the detectors. We disclose that failure mode is largely associated with the output format of a model rather than its architecture family. Of the three, only omission is visible to a downstream system; displacement and substitution return confident, well-formed boxes, making them potentially more dangerous failures. Further, we find that, under distribution shift, VLMs that primarily failed by omission instead shift to failing more by displacement/substitution. Thus, a grounding VLM can inherit the detector's vulnerability while showing the possibility to adopt more dangerous failure modes. Our code and data are available at https://github.com/ericliang1/physical-adversarial-robustness.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Adversarial system, Robustness (evolution), Computer science, Artificial intelligence, Computer security, Vulnerability (computing)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange