Newest papers

Perceives but Doesn't Brake: A Perception-Action Binding Gap in Reasoning Driving VLAs

Matteo Cercola, Katie Z Luo, Simone Formentin, Marco Pavone · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28184 · cited by 0 works (OpenAlex) · open-access PDF

German

Wir charakterisieren dieses Versagen als eine Wahrnehmungs-Aktionsbindungslücke: Gefahreninformationen sind innerhalb des Modells verfügbar, üben jedoch unzureichende Kontrolle über die Trajectory-Generation des Modells aus. Durch die Verwendung von simulierten Kontrafaktualitäten und logged-Driving-Wiederholung zeigen wir, dass Gefahren in der Straße tatsächlich in der visuellen Darstellung kodiert sind, aber Längspläne werden sogar von der Ego-Geschichte dominiert und reagieren nur schwach auf Gefahrenproximität oder die eigene Argumentationsspur des Modells. Weiterführende Analyse zeigt, dass der Trajectory-Generator in der Lage ist, zu bremsen, aber manchmal dies aufgrund der Wahrnehmungs-Aktionsbindungslücke nicht tut. Ohne eine einfache Brem

Machine translation (Meta M2M100); the English original follows.

Abstract

Reasoning-augmented vision-language-action (VLA) driving models can perceive and reason about safety-critical hazards yet fail to translate that evidence into safe actions. We characterize this failure as a perception-action binding gap: hazard information is available inside the model but exerts insufficient control over trajectory generation. Using matched simulated counterfactuals and logged driving replay, we show that in-path hazards are actually encoded in the visual representation, but longitudinal plans are dominated by ego-speed history and respond only weakly to hazard proximity or the model's own reasoning trace. Further analysis shows that the trajectory generator is capable of braking but sometimes fails to do so due to the perception-action binding gap. Without retraining or modifying any weights, we inject a hazard-conditioned energy into the frozen flow-matching sampler of Alpamayo-1.5, restoring the missing coupling at generation time. With exact obstacle geometry, guidance cuts footprint intrusion from 58.1% to 14.5% and recovers safe braking whenever sufficient stopping distance remains, eliminating collisions entirely in our braking-relevant closed-loop battery. It also reverses the diagnosed dependence of the plan, from tracking ego speed to responding to obstacle range. These results separate hazard perception from hazard control in driving VLAs, show that unsafe behavior can arise even when the model has both the evidence and the ability to act on it, and propose a simple fix for bridging this binding gap.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Trajectory, Computer science, Hazard, Bayesian probability, Control theory (sociology), Obstacle

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange