Kimi k1.5: Scaling Reinforcement Learning with LLMs

Understand

Language model pretraining with next token prediction has proved effective for scaling compute but is limited to the amount of available training data.

• Scaling reinforcement learning (RL) unlocks a new axis for the continued improvement of artificial intelligence, with the promise that large language models (LLMs) can scale their training data by learning to explore with rewards.
• However, prior published work has not produced competitive results.
• In light of this, we report on the training practice of Kimi k1.5, our latest multi-modal LLM trained with RL, including its RL training techniques, multi-modal data recipes, and infrastructure optimization.