Newest papers

Retrieval-Augmented Online Textual Memory for Episodic Memory Video Question Answering

Raffaele Calì, Giuseppe Lando, Rosario Forte, Antonino Furnari · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31284 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

O fluxo visual deve ser comprimido em uma representação de memória que pode ser pesquisada de forma eficiente, preservando ainda a evidência necessária para responder a pergunta. Com base em um quadro de memória de episódios em linha em que os vídeos de primeira pessoa são convertidos em cabeçalhos temporalmente baseados, este trabalho estuda sistematicamente o papel da recuperação na resposta a perguntas de vídeo de memória episódica. Avaliamos como diferentes unidades de recuperação, formulações de consulta, orçamentos de memória específicos de contexto e estratégias de rearranjo afetam a escolha múltipla de respostas em QAEgo4D-Closed, isolando a recuperação como a variável principal, mantendo a evidência subjacente gerada na memória. Nossa melhor configuração, Parent, combina subchunks de

Machine translation (Meta M2M100); the English original follows.

Abstract

Long egocentric videos pose a central challenge for multimodal language models: the visual stream must be compressed into a memory representation that can be searched efficiently while still preserving the evidence needed for question answering. Building on an online textual-memory framework in which first-person videos are converted into temporally grounded captions, this work systematically studies the role of retrieval in episodic-memory video question answering. We evaluate how different retrieval units, query formulations, context budgets, and reranking strategies affect multiple-choice answering on QAEgo4D-Closed, isolating retrieval as the main variable while keeping the underlying generated memory fixed. Our best configuration, Parent, matches fine-grained caption subchunks, then provides their full parent captions to the reasoner. It achieves 56.4 % accuracy, matching strong full-memory and retrieval baselines while exposing only a small fraction of the textual memory at question time. More importantly, our analysis shows that better temporal coverage does not necessarily translate into better answers: increasing the retrieval budget improves temporal Hit@K and Recall@K, but can reduce accuracy by introducing visually plausible distractors and uncertain caption evidence. These findings suggest that retrieval-augmented egocentric memory should not be optimized only for evidence recall, but also for the reliability, specificity, and answer-discriminative value of the retrieved context. The study motivates more structured episodic memories that move beyond raw caption retrieval toward evidence-aware representations for long-form egocentric reasoning.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Question answering, Computer science, Episodic memory, Matching (statistics), Context (archaeology), Representation (politics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange