Jielong Tang, Ling-An Zeng, Xujie Yuan, Xiao Dong, Zhiqian Xia, Qingfeng Li, Shixiong Kai, Kaichao Liang, Rong Yao, Jian Yin · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.29995 · cited by 0 works (OpenAlex) · open-access PDF
La mémoire à long terme pour les agents multimodaux reste un défi, car elle doit conserver des preuves visuelles fines tout en suivant l'évolution des entités à travers les sessions. Les mémoires de vérification à base d'abstraction existantes compriment efficacement des images en texte, rejetant souvent des détails discriminatoires visuellement, tandis que les mémoires de conservation de pixels conservent des observations brutes mais luttent pour associer de manière fiable diverses observations à des identités d'entités persistantes et suivre leurs états évolutifs. Pour traiter ces limitations, nous proposons MHG-Mem, une mémoire hypergraphique multimodale qui organise conjointement l'identité de l'entité, l'évolution de l'état et la provenance à des pre
Machine translation (Meta M2M100); the English original follows.
Long-term memory for multimodal agents remains challenging, as it must preserve fine-grained visual evidence while tracking entity evolution across sessions. Existing abstraction-based memories compress images into text, often discarding visually discriminative details, whereas pixel-preserving memories retain raw observations but struggle to reliably associate diverse observations with persistent entity identities and track their evolving states. To address these limitations, we propose MHG-Mem, a multimodal hypergraph memory that jointly organizes entity identity, state evolution, and provenance to visual evidence. Specifically, MHG-Mem represents distinct entity mentions and their observed facts as individual nodes, grouping identity-consistent mentions and facts through occurrence hyperedges. It then synthesizes aspect-specific evolution states from these grouped observations and links them to supporting entities, facts, and source evidence through evolutionary and evidential hyperedges. To navigate this structured memory, we introduce hypergraph-based agentic search, which adaptively selects an entry layer, traverses typed hyperedges across memory levels, and inspects source observations when fine-grained verification is required. These designs support both detailed visual verification and reasoning over cross-session state changes. Extensive experiments across three benchmarks demonstrate the effectiveness of MHG-Mem, improving performance by 8.20-9.98% over the strongest baseline.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Discriminative model, Hypergraph, Artificial intelligence, State (computer science), Natural language processing
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.29995, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange