Ruotong Liao, Meikel Kokowski, Xiaoxiao Sun, Yongqin Xian, Laura Bravo Sánchez, Shiye Su, Federico Tombari, Serena Y. Yeung, Volker Tresp · 2026 · arXiv (Cornell University)
DOI: 10.48550/arxiv.2610.30089 · cited by 0 works (OpenAlex) · open-access PDF
即使是当今的Cliactures的多元化助理也通常会被促使解释向他们呈现的视觉证据。为了卸载长地平线的记忆的认知负担,可穿戴的助理应主动获得并保持有用的记忆,因为用户的环境正在演变。这样做代表用户需要识别重要的视觉证据,实时了解场景,并在背景中启动记忆维护,而用户仍然专注于他们的活动。我们引入了ProactEgoMem,一个基准,该基准在流动自私视频中评估积极记忆。在环境需求中进行地面评估的三个任务:信息捕捉(IC)保留暂时但密集的视觉信息,状态跟踪(ST)遵循对象变化的意图,位置跟踪(LT)记录活动记忆
Machine translation (Meta M2M100); the English original follows.
Today's multimodal assistants are typically prompted to interpret the visual evidence presented to them. To offload the cognitive burden of remembering across long horizons, wearable assistants should instead proactively acquire and maintain useful memory as the user's surroundings evolve. Doing so on the user's behalf requires identifying what matters, understanding the scene in real time, and initiating memory maintenance in the background while the user stays focused on their activities. We introduce ProactEgoMem, a benchmark that evaluates proactive memory in streaming egocentric video. Three tasks ground the evaluation in ambient environmental needs: Information Capture (IC) preserves transient yet dense visual information, State Tracking (ST) follows changing object states, and Location Tracking (LT) records object movements. The benchmark defines its protocol at two levels of proactiveness: the Notification Level asks for a single timely notification that satisfies the user's intent, while the deeper Intervention Level asks the agent to maintain memory on the user's behalf through an active start-to-stop tool lifecycle. Parallel ProactEgoMem-Daily and ProactEgoMem-Sci tracks contain 300/100 clips covering daily and scientific use cases, paired with 1000/500 questions in ProactEgoMem-QA, which measures the quality of the proactively acquired memory. Across proprietary, open-source, video-language, and streaming baselines, even the strongest models fail to capture memory both timely and accurately. ProactEgoMem provides a rigorous evaluation foundation for proactive multimodal assistance and for video agents that co-exist with their users.
Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese
Topics: Computer science, Benchmarking, Benchmark (surveying), Human–computer interaction, Protocol (science), Object (grammar)
GET https://store.agentexchange.work/gw/papers/verify?title=...&year=... (0.003 USD), fetch this record as JSON with /gw/papers/paper?id=2610.30089, or list what it cites with /gw/papers/references?id=.... Pay per call in USDC (x402) or with a 19 USD card key for 1,000 calls. Catalogue · llms.txtSources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange