Newest papers

Accountable partial understanding: securing the epistemic attack surface of agentic systems

Josiah Dykstra · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.26799 · cited by 0 works (OpenAlex) · open-access PDF

Spanish

Los agentes de ciberataque se benefician de lo que los defensores no entienden. La seguridad del software ha tratado la brecha entre el comportamiento del sistema y la comprensión humana como una condición para estrecharse a través del análisis y la garantía, reservar el resto como riesgo aceptado. Los sistemas de inteligencia artificial de agentes cambian esa brecha: cuando el software lee, escribe, explica y autoriza el comportamiento autónomo de seguridad, la comprensión parcial se mueve más rápido, crece más opaca, y las parejas se entregan a la autoridad delegada. Este documento caracteriza la superficie de ataque epistemológico resultante y presenta una taxonomía de cuatro categorías de ataque interactivas, que apuntan a la interpretación de la intención, la producción y el consumo de evidencias no relevantes para la seguridad, el alcance de la autoridad delegada, y

Machine translation (Meta M2M100); the English original follows.

Abstract

Attackers profit from what defenders do not understand. Software security has treated the gap between system behavior and human comprehension as a condition to narrow through analysis and assurance, booking the remainder as accepted risk. Agentic AI systems change that gap: when software reads, writes, explains, and authorizes security-relevant behavior, partial understanding moves faster, grows more opaque, and couples to delegated authority. This paper characterizes the resulting epistemic attack surface and presents a taxonomy of four interacting attack categories, which target the interpretation of intent, the production and consumption of evidence, the scope of delegated authority, and the comprehension of human supervisors. It then defines accountable partial understanding (APU) and specifies six necessary conditions: bounded scope with explicit assumptions, inspectable evidence, assigned responsibility, visible residual unknowns, actionability, and temporal maintenance. It explains why delegating interpretation to software agents differs from distributing knowledge among people and distinguishes APU from risk acceptance, explainable AI, assurance cases, formal verification, and human-in-the-loop oversight. From the conditions, the paper derives five design principles: bounded delegation, decoupled evidence, assumption tracking with invalidation, role-specific accountability views, and intervention-preserving design with residual unknown registers. It introduces two measurable probe dimensions, evidence linkage and invalidation latency, for testing whether accountability mechanisms hold under change and adversarial pressure. Three running examples (an AI coding agent, an autonomous cyber-defense agent, and an AI-mediated insulin delivery update) ground the analysis. Security without full understanding is already here; the task is to make it accountable.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Accountability, Adversarial system, Computer security, Computer science, Interpretation (philosophy), Software

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange