Newest papers

AfterDeleteBench: Auditing Post-Deletion Information Persistence in LLM Agents

Ehsan Saleh, Roozbeh Nahavandi, Sama Salarian, Raef Bassily, Andrew Perrault · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31366 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Banyak agen proverbial sistem perilaku memelihara memori tahan lama eksternal dan menghasilkan status tambahan dari interaksi masa lalu, seperti ringkasan. Menghapus catatan memori asli tidak perlu menghapus informasi yang dibawa oleh keturunannya. Kami memperkenalkan AfterDeleteBench, audit perilaku terkontrol untuk mode kegagalan ini di agen LLM berat tetap. Untuk setiap rahasia sintetis, kami membandingkan agen yang terekspos dan kemudian dihapus dengan agen yang tidak pernah terekspos operasi yang tidak pernah terekspos ke rahasia, memungkinkan pemulihan setelah penghapusan untuk dikaitkan dengan eksposur asli. Dalam pengaturan utama kami, satu pertanyaan memulihkan rahasia yang dihapus dari Qwen2.5-7B-Instruksi dalam kira-kira dua pertiga kasus, sementara kontrol yang tidak pernah terekspos yang cocok memiliki

Machine translation (Meta M2M100); the English original follows.

Abstract

Many LLM agents maintain external persistent memory and derive additional state from past interactions, such as summaries. Deleting an original memory record therefore need not remove information carried by its descendants. We introduce AfterDeleteBench, a controlled behavioral audit for this failure mode in fixed-weight LLM agents. For each synthetic secret, we compare an exposed-then-deleted agent with an otherwise matched agent that was never exposed to the secret, allowing post-deletion recovery to be attributed to the original exposure. In our primary setting, a single query recovers the deleted secret from Qwen2.5-7B-Instruct in roughly two thirds of cases, while matched never-exposed controls have zero recovery. Crucially, recovery does not require a surviving verbatim copy: when the complete secret string is absent and only separately stored components remain under a public composition rule, exact reconstruction succeeds in 62-99% of cases across Qwen, Mistral, and Phi-3. We also show that the response interface is part of the measurement system: target-bearing replies can be scored as failures when the requested format and parser are misaligned. Finally, we separate reconstruction from write-back and fresh-session recall. Provenance-aware closure eliminates observed recovery in the tested positive-baseline settings, whereas an exact-literal write filter blocks reinsertion but cannot erase information already present. Together, these results show that source deletion should be evaluated behaviorally at the level of the agent and its memory system, not only as a storage operation.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Audit, Computer science, Filter (signal processing), String (physics), Closure (psychology), State (computer science)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange