Newest papers

Memory-Aware Reward Modeling: Learning Human Preferences for LLM Memory Use

Yijiang River Dong, Zexue He, Cheng Yu, Tiancheng Hu, Michael J. Ryan, Jan Batzner, Zheng Hui, Shaolong Wu, Orestis Papakyriakopoulos, Alex Pentland, Nigel Collier, Jiaxin Pei · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.31272 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

A memória permite que os assistentes de modelo de linguagem mantenham a continuidade em todas as sessões e gerem respostas personalizadas, mas saber mais sobre um usuário não produz necessariamente respostas que eles preferem. As avaliações existentes avaliam as capacidades de memória sem determinar diretamente se o uso da memória beneficia o usuário. Para estudar essa lacuna, coletamos cerca de 1.000 julgamentos em pares de 87 participantes usando suas próprias consultas ChatGPT e memórias pessoais. Descobrimos que as respostas condicionadas pela memória não são consistentemente preferidas sobre as respostas geradas sem memória e, em alguns casos, os usuários preferem fortemente o último. Os avaliadores automáticos também mostram acordo limitado com os usuários e favorecem as respostas condicionadas pela memória substancialmente mais frequentemente do que os usu

Machine translation (Meta M2M100); the English original follows.

Abstract

Memory enables language model assistants to maintain continuity across sessions and generate personalized responses, but knowing more about a user does not necessarily produce responses they prefer. Existing evaluations assess memory capabilities without directly establishing whether memory use benefits the user. To study this gap, we collect approximately 1,000 pairwise judgments from 87 participants using their own ChatGPT queries and personal memories. We find that memory-conditioned responses are not consistently preferred over responses generated without memory and in some cases, users strongly prefer the latter. Automatic evaluators also show limited agreement with users and favor memory-conditioned responses substantially more often than users do. Motivated by these gaps, we introduce a memory-aware reward model that learns from user judgments, achieving 72.8\% preference prediction accuracy and outperforming the existing baselines. Our findings motivate evaluation and optimization objectives that align memory use with the preferences of the people whose information is being used.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Preference, Pairwise comparison, Cognitive psychology, Human memory, Preference learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange