Newest papers

A Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM Era

Eric Liang, Alan Luo, Kaiwen Yuan · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29196 · cited by 0 works (OpenAlex) · open-access PDF

Russian

Недавние уязвимости в виде VLM позволяют создавать уязвимости с использованием языка и генерирующего декодирования, демонстрируя в значительной степени улучшенные возможности обнаружения нулевого удара и открытого словарного потенциала. В результате VLM быстро развертываются на краю в опасно-критических системах реального мира. Тем не менее, физическая уязвимость, которая используется на земле, не была систематически изучена, и фундаментальные различия в выходе с нашими современными детекторами объектов создают методологические проблемы для справедливой оценки. Чтобы сравнить две семьи напрямую, мы оцениваем каждую выбранную представительскую модель по одному исходу: может ли враждебный шаг на земле обнаружить замену модели. Мы сообщаем об этом, внедряя

Machine translation (Meta M2M100); the English original follows.

Abstract

Recent advancements in Vision-Language Models (VLMs) enable end-to-end visual grounding via language prompts and generative decoding, demonstrating largely improved zero-shot and open-vocabulary detection capabilities. As a result, VLMs are rapidly being deployed at the edge in safety-critical real-world systems. However, the physical adversarial robustness of edge VLMs has not been systematically studied, and the fundamental differences in output with modern object detectors create methodological challenges for fair evaluation. To compare the two families directly, we evaluate every selected representative model on one outcome: whether an adversarial patch worn on a person evades detection of the model. We report this by introducing a unified metric, conditional attack success rate (cASR), counted only on images the model still localizes under a matched gray occluder, and partition every failure into omission (no box), substitution (a box on the patch), and displacement (a box on neither). We benchmark the physical adversarial vulnerability of compact grounding VLMs against a range of object detectors, and find that the VLMs can be as vulnerable as the detectors. We disclose that failure mode is largely associated with the output format of a model rather than its architecture family. Of the three, only omission is visible to a downstream system; displacement and substitution return confident, well-formed boxes, making them potentially more dangerous failures. Further, we find that, under distribution shift, VLMs that primarily failed by omission instead shift to failing more by displacement/substitution. Thus, a grounding VLM can inherit the detector's vulnerability while showing the possibility to adopt more dangerous failure modes. Our code and data are available at https://github.com/ericliang1/physical-adversarial-robustness.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Adversarial system, Robustness (evolution), Computer science, Artificial intelligence, Computer security, Vulnerability (computing)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange