Fetching the paper…
Reading the bibliography…
We revisit policy-gradient optimization for Large Language Models (LLMs) from a single-stream perspective.
Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J Williams · 1992
Earlier work this paper cites.
Variance reduction techniques for gradient estimates in reinforcement learning
Evan Greensmith, Peter L Bartlett, and Jonathan Baxter · 2004
Earlier work this paper cites.
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver · 2015
Earlier work this paper cites.
Proximal policy optimization algorithms
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov · 2017
Earlier work this paper cites.
IMPALA: Scalable distributed deep-RL with importance weighted actor-learner architectures
Lasse Espeholt, Hubert Soyer, Remi Munos, Karen Simonyan, Vlad Mnih, Tom Ward, Yotam Doron, Vlad Firoiu, Tim Harley, Iain Dunning, et al · 2018
Earlier work this paper cites.
Reinforcement learning: An introduction
Richard S Sutton and Andrew G Barto · 2018
Earlier work this paper cites.
What matters in on-policy reinforcement learning? A large-scale empirical study
Marcin Andrychowicz, Anton Raichuk, Piotr Stańczyk, Manu Orsini, Sertan Girgin, Raphael Marinier, Léonard Hussenot, Matthieu Geist, Olivier Pietquin, Marcin Michalski, et al · 2020
Earlier work this paper cites.
Evaluating large language models trained on code
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al · 2021
Earlier work this paper cites.
Self-consistency improves chain of thought reasoning in language models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou · 2023
Earlier work this paper cites.
Back to basics: Revisiting reinforce style optimization for learning from human feedback in LLMs
Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker · 2024
Earlier work this paper cites.
InfAlign: Inference-aware language model alignment
Ananth Balashankar, Ziteng Sun, Jonathan Berant, Jacob Eisenstein, Michael Collins, Adrian Hutter, Jong Lee, Chirag Nagpal, Flavien Prost, Aradhana Sinha, et al · 2024
Earlier work this paper cites.
Tulu 3: Pushing frontiers in open language model post-training
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al · 2024
Earlier work this paper cites.
DeepSeekMath: Pushing the limits of mathematical reasoning in open language models
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al · 2024
Earlier work this paper cites.
HybridFlow: A flexible and efficient RLHF framework
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu · 2024
Cited alongside, same era.
POLARIS: A post-training recipe for scaling reinforcement learning on advanced reasoning models, 2025
Chenxin An, Zhihui Xie, Xiaonan Li, Lei Li, Jun Zhang, Shansan Gong, Ming Zhong, Jingjing Xu, Xipeng Qiu, Mingxuan Wang, and Lingpeng Kong · 2025
Cited alongside, same era.
MathArena: Evaluating LLMs on uncontaminated math competitions
Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev · 2025
Cited alongside, same era.
Accelerating RL for LLM reasoning with optimal advantage regression
Kianté Brantley, Mingyu Chen, Zhaolin Gao, Jason D Lee, Wen Sun, Wenhao Zhan, and Xuezhou Zhang · 2025
Cited alongside, same era.
The entropy mechanism of reinforcement learning for reasoning language models
Understanding Tool-Integrated Reasoning
Heng Lin and Zhongwen Xu · 2025
Closest in time.
DeepScaleR: Surpassing o1-Preview with a 1.5B model by scaling RL
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica · 2025
Closest in time.
MiniMax-M1: Scaling test-time compute efficiently with lightning attention
Team MiniMax · 2025
Closest in time.
Can prompt difficulty be online predicted for accelerating RL finetuning of reasoning models?
Yun Qu, Qi Wang, Yixiu Mao, Vincent Tao Hu, Björn Ommer, and Xiangyang Ji · 2025
Closest in time.
Team Qwen · 2025
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al · 2025
Cited alongside, same era.
DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
Team DeepSeek · 2025
Cited alongside, same era.
ReTool: Reinforcement learning for strategic tool use in LLMs
Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong · 2025
Cited alongside, same era.
Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao · 2025
Cited alongside, same era.
Beyond ten turns: Unlocking long-horizon agentic search with large-scale asynchronous RL
Jiaxuan Gao, Wei Fu, Minyang Xie, Shusheng Xu, Chuyi He, Zhiyu Mei, Banghua Zhu, and Yi Wu · 2025
Cited alongside, same era.
On-policy RL with optimal reward baseline
Yaru Hao, Li Dong, Xun Wu, Shaohan Huang, Zewen Chi, and Furu Wei · 2025
Cited alongside, same era.
SkyWork Open Reasoner 1 technical report
Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, et al · 2025
Cited alongside, same era.
REINFORCE++: An efficient RLHF algorithm with robustness to both prompt and reward models
Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen · 2025
Cited alongside, same era.
Closest in time.
Seed1.5-thinking: Advancing superb reasoning models with reinforcement learning
Team Seed · 2025
Closest in time.
rStar2-Agent: Agentic Reasoning Technical Report
Ning Shang, Yifei Liu, Yi Zhu, Li Lyna Zhang, Weijiang Xu, Xinyu Guan, Buze Zhang, Bingcheng Dong, Xudong Zhou, Bowen Zhang, et al · 2025
Closest in time.
OctoThinker: Mid-training incentivizes reinforcement learning scaling
Zengzhi Wang, Fan Zhou, Xuefeng Li, and Pengfei Liu · 2025
Closest in time.
Bo Wu, Sid Wang, Yunhao Tang, Jia Ding, E1yk Helenowski, Liang Tan, Tengyu Xu, Tushar Gowda, Zhengxing Chen, Chen Zhu, et al · 2025
Closest in time.
Agents play thousands of 3D video games
Zhongwen Xu, Xianliang Wang, Siyi Li, Tao Yu, Liang Wang, Qiang Fu, and Wei Yang · 2025
Closest in time.
DAPO: An open-source LLM reinforcement learning system at scale
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al · 2025
Closest in time.
GLM-4.5: Agentic, reasoning, and coding (ARC) foundation models
Team Zhipu · 2025
Closest in time.