Newest papers

ProactEgoMem: Benchmarking Proactive Memory in Streaming Egocentric Video

Ruotong Liao, Meikel Kokowski, Xiaoxiao Sun, Yongqin Xian, Laura Bravo Sánchez, Shiye Su, Federico Tombari, Serena Y. Yeung, Volker Tresp · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.30089 · cited by 0 works (OpenAlex) · open-access PDF

Indonesian

Untuk mengecilkan beban kognitif dari mengingat melalui cakrawala yang panjang, asisten yang dapat dipakai harus secara proaktif memperoleh dan mempertahankan memori yang berguna saat lingkungan pengguna berkembang. Melakukannya atas nama pengguna membutuhkan mengidentifikasi apa yang penting, memahami adegan secara real time, dan memulai pemeliharaan memori di latar belakang sementara pengguna tetap fokus pada aktivitas mereka. Kami memperkenalkan ProactEgoMem, sebuah referensi yang mengevaluasi memori proaktif dalam streaming video egocentris. Tiga tugas penilaian dasar dalam kebutuhan lingkungan membutuhkan: Informasi Capture (IC) mempertahankan informasi pro-stop namun padat visual, State Tracking (ST) mengikuti keadaan objek yang berubah, dan Lokasi Tracking (LT) mencatat gerakan memori objek aktif. Benchmark ini mendefinisikan protokolnya pada dua tingkat proaktifitas ilmiah:

Machine translation (Meta M2M100); the English original follows.

Abstract

Today's multimodal assistants are typically prompted to interpret the visual evidence presented to them. To offload the cognitive burden of remembering across long horizons, wearable assistants should instead proactively acquire and maintain useful memory as the user's surroundings evolve. Doing so on the user's behalf requires identifying what matters, understanding the scene in real time, and initiating memory maintenance in the background while the user stays focused on their activities. We introduce ProactEgoMem, a benchmark that evaluates proactive memory in streaming egocentric video. Three tasks ground the evaluation in ambient environmental needs: Information Capture (IC) preserves transient yet dense visual information, State Tracking (ST) follows changing object states, and Location Tracking (LT) records object movements. The benchmark defines its protocol at two levels of proactiveness: the Notification Level asks for a single timely notification that satisfies the user's intent, while the deeper Intervention Level asks the agent to maintain memory on the user's behalf through an active start-to-stop tool lifecycle. Parallel ProactEgoMem-Daily and ProactEgoMem-Sci tracks contain 300/100 clips covering daily and scientific use cases, paired with 1000/500 questions in ProactEgoMem-QA, which measures the quality of the proactively acquired memory. Across proprietary, open-source, video-language, and streaming baselines, even the strongest models fail to capture memory both timely and accurately. ProactEgoMem provides a rigorous evaluation foundation for proactive multimodal assistance and for video agents that co-exist with their users.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Benchmarking, Benchmark (surveying), Human–computer interaction, Protocol (science), Object (grammar)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange