Newest papers

From RL Papers to Framework Code: Inclusion and Review in Seven Open-Source Training Frameworks

Taiheng Pan · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28027 · cited by 0 works (OpenAlex) · open-access PDF

Korean

프레임 워크를 통해 교육 프레임 워크를 통해 언어 모델에 대한 공공 프레임 워크 학습 방법을 사용하여 프레임 워크가 포함하는 방법은 사용자에 대한 하나의 구성 변화입니다. 우리는 얼마나 자주 이것이 발생하는지 측정합니다. 우리는 프레임 워크에 대한 7 개의 공개 소스 프레임 워크를 사용하여 커미 스토리를 통해 RL 포스트 트레이닝을 추적하고 6,939 개의 프레임 워크에 연결합니다. 프레임 워크를 포함하는 1.598 개의 논문에서 일반적인 RL 트레이닝 방법을 제안하는 것보다, 37 개는 적어도 하나의 프레임 워크에 도달했습니다. 프레임 워크의 5 분기 제안을 통해 180 일 이내에 포함 된 숫자는 5와 7 사이에 머물

Machine translation (Meta M2M100); the English original follows.

Abstract

Practitioners use reinforcement learning methods for language models through training frameworks, so a method that a framework includes is one configuration change away for its users. We measure how often this happens. We trace seven open-source frameworks for RL post-training through their commit histories and link them to 6,939 screened arXiv papers from January 2025 to June 2026. Of 1,598 papers that propose a general RL training method, 37 reached at least one framework. Across five quarterly cohorts of papers, the number included within 180 days stayed between 5 and 7 while the cohorts grew from 72 to 386 papers. Compared with papers from the same quarters, included papers more often trained policies of 30B parameters or more, at 41 against 10 percent, and more often changed a coefficient, clip, mask or weight in the existing loss, at 29 against 9 percent. Both gaps hold without papers from the companies that maintain the frameworks, while code release was equally common in both groups. The frameworks' issues and pull requests show that few methods are turned away in public. More than nine in ten of the general methods no framework included have no thread there that cites or names them, and many proposals that do appear stay open without a reply. Two further records, the frameworks' issues and the technical reports of 13 large open-weight models, show that users rarely report training failures in public and that the reports state 8 of the 13 widely included methods. We release the data as a tracker that links training failures to framework implementations and lists each framework's contribution rules and earlier proposals.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Commit, CITES, Computer science, Inclusion (mineral), Reinforcement learning, Code (set theory)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange