Newest papers

From RL Papers to Framework Code: Inclusion and Review in Seven Open-Source Training Frameworks

Taiheng Pan · 2026 · arXiv (Cornell University)

DOI: 10.48550/arxiv.2610.28027 · cited by 0 works (OpenAlex) · open-access PDF

Portuguese

Os praticantes de relatórios usam métodos de aprendizagem de dados de reforço para modelos de linguagem através de frameworks de treinamento, de modo que um método que um framework inclui é uma mudança de configuração para seus usuários. Medimos quantas vezes isso acontece. Nós rastreamos sete frameworks de código aberto para o pós-treinamento de RL através de suas histórias de compromisso e os vinculamos a 6.939 papéis arXiv rastreados de janeiro de 2025 a junho de 2026. De 1.598 papéis que propõem um método de treinamento geral de RL, 37 atingiram pelo menos um framework. Ao longo de cinco cohortes trimestrais de propostas de papéis, o número incluído dentro de 180 dias permaneceu entre 5 e 7 enquanto as cohortes cresceram de 72 a 386 papéis. Em comparação com os pap

Machine translation (Meta M2M100); the English original follows.

Abstract

Practitioners use reinforcement learning methods for language models through training frameworks, so a method that a framework includes is one configuration change away for its users. We measure how often this happens. We trace seven open-source frameworks for RL post-training through their commit histories and link them to 6,939 screened arXiv papers from January 2025 to June 2026. Of 1,598 papers that propose a general RL training method, 37 reached at least one framework. Across five quarterly cohorts of papers, the number included within 180 days stayed between 5 and 7 while the cohorts grew from 72 to 386 papers. Compared with papers from the same quarters, included papers more often trained policies of 30B parameters or more, at 41 against 10 percent, and more often changed a coefficient, clip, mask or weight in the existing loss, at 29 against 9 percent. Both gaps hold without papers from the companies that maintain the frameworks, while code release was equally common in both groups. The frameworks' issues and pull requests show that few methods are turned away in public. More than nine in ten of the general methods no framework included have no thread there that cites or names them, and many proposals that do appear stay open without a reply. Two further records, the frameworks' issues and the technical reports of 13 large open-weight models, show that users rarely report training failures in public and that the reports state 8 of the 13 widely included methods. We release the data as a tracker that links training failures to framework implementations and lists each framework's contribution rules and earlier proposals.

Read in: English · Spanish · Chinese · Hindi · Arabic · Portuguese · French · German · Japanese · Russian · Korean · Indonesian · Vietnamese

Topics: Commit, CITES, Computer science, Inclusion (mineral), Reinforcement learning, Code (set theory)

Sources: OpenAlex (CC0), arXiv, Crossref. Agent Exchange