Newest papers

Self-Credit: Credit Assignment for Long-Horizon Agentic RL via Intra-group Rescaling

Fangcong Yin, Ming Yin, Tanvi Bhandarkar, Aashu Singh, Eryk Helenowski, Shlok Kumar Mishra, Saghar Hosseini, Karthik Abinav Sankararaman, Greg Durrett, Dinkar Juyal · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28388 · cited by 0 works (OpenAlex) · open-access PDF

Chinese

一般来说,在RLVR中,自己选择的代理人通常是RLVR中挑选的,但典型的RLVR奖励的稀缺性质使得信用分配具有挑战性,特别是在长期任务中。我们提出了自我信贷,一个端到端的代理RL方法,它会创建密集的监督信号,从稀缺的结果奖励中使用自我生成的信贷分配,通过群体内部重新分配。自我信贷构建了基于组的RL和三个组成部分的挑战性。 首先,在其部署过程中,该模型政策会生成引用到包含关键证据的先前转换的评论的汇总,这将作为自我生成的信贷奖励。 其次,我们通过群体内部重新分配来改进这种模式生成的信贷,这将对同一

Machine translation (Meta M2M100); the English original follows.

Abstract

LLM agents are commonly trained with RLVR, but the sparse nature of typical RLVR rewards makes credit assignment challenging, especially for long-horizon tasks. We propose Self-Credit, an end-to-end agentic RL method that creates dense supervision signals from sparse outcome rewards using self-generated credit assignment via intra-group rescaling. Self-Credit builds on group-based RL with three components. First, during its rollouts, the policy model generates summaries with citations to earlier turns containing critical evidence, which serve as self-generated credit. Second, we refine this model-generated credit with intra-group rescaling, which compares cited actions across successful and failed rollouts for the same prompt. This approach uses group-level agreement and disagreement to remove undeserved credit and further emphasize credit assigned to critical actions. Third, advantage reweighting uses the rescaled credit weights to update the advantages of the cited turns. On BrowseComp-Plus, Self-Credit improves over the base models by 37.0 and 36.2 points on Qwen3 8B and 32B respectively, and outperforms the strongest agentic RL baselines by 8.5 and 5.6 points, respectively. Self-Credit also improves over the strongest baselines by 4.9 points on GAIA and 4.7 points on xBench-DeepSearch. Analyses show that self-generated credit provides useful signals for selecting critical actions, and that intra-group rescaling, especially on failed rollouts, is important across model scales.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Computer science, Base (topology), Econometrics, Outcome (game theory), Artificial intelligence, Machine learning

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange