Newest papers

Uni-Mem: Learning a Long-Horizon Agentic Memory Manager with Unified Online Policy Optimization

Zhiqian Xia, Jielong Tang, Ling-An Zeng, Shixiong Kai, Rong Yao, Hao Kong, Xiao Dong, Wei Wang, Mingxuan Yuan, Siyu Xia, Haifeng Xia · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.29671 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

A memória de agente de longo prazo está mudando de fluxos de trabalho pré-definidos para políticas aprendíveis para construção e recuperação de memória. Os métodos existentes baseados em aprendizagem otimizam a construção sozinho ou combinam o aprendizado de reforço on-line (RL) do lado de recuperação com a supervisão de construção off-line, limitando a coadaptação entre as duas fases. Neste artigo, exploramos a RL on-line de ponta a ponta para um gerente de memória unificado (Uni-Mem) que aprende ambas as capacidades sob uma política compartilhada. No entanto, a otimização conjunta com os algoritmos RL existentes, como o GRPO, enfrenta comparações de grupo confusas e feedback escasso sobre trajetórias de gerenciamento de memória longa. Para este fim,

Machine translation (Meta M2M100); the English original follows.

Abstract

Long-term agent memory is shifting from predefined workflows toward learnable policies for memory construction and retrieval. Existing learning-based methods either optimize construction alone or combine retrieval-side online reinforcement learning (RL) with offline construction supervision, limiting co-adaptation between the two stages. In this paper, we explore end-to-end online RL for a unified memory manager (Uni-Mem) that learns both capabilities under a shared policy. However, joint optimization with existing RL algorithms such as GRPO faces confounded group comparisons and sparse feedback over long memory-management trajectories. To this end, we propose Stage-Decoupled GRPO, which alternates construction and retrieval optimization while controlling variation from the other stage within each rollout group, yielding stage-specific learning signals. We further introduce Event-Wise Credit Assignment (ECA), which localizes feedback to responsible events and propagates event-level advantages to their constituent operations. Experiments on LoCoMo and LongMemEval show that Uni-Mem outperforms the strongest RL-based memory, demonstrating the effectiveness of jointly learning memory construction and retrieval via Stage-Decoupled GRPO and ECA.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Reinforcement learning, Workflow, Artificial intelligence, Limiting, Scheme (mathematics)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange