Not All Correct Answers Are Equal: Why Your Distillation Source Matters

Chemistry-qa

Microsoft · 2021

Earlier work this paper cites.

Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks, 2022

Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Gary Lai, et al · 2022

Earlier work this paper cites.

Let’s verify step by step, 2023

Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe · 2023

Earlier work this paper cites.

Logic-701: A benchmark dataset for logical reasoning in english and russian

hivaze · 2023

Earlier work this paper cites.

Openhermes 2.5: An open dataset of synthetic data for generalist llm assistants, 2023

Teknium · 2023

Earlier work this paper cites.

Enhancing chat language models by scaling high-quality instructional conversations, 2023

Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou · 2023

Earlier work this paper cites.

Openorca: An open dataset of gpt augmented flan reasoning traces

Wing Lian, Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium" · 2023

Earlier work this paper cites.

Fine flan: Seqio to parquet so you don’t have to

Bleys Goodson · 2023

Earlier work this paper cites.

American invitational mathematics examination - aime

MAA · 2024

Earlier work this paper cites.

Livecodebench: Holistic and contamination free evaluation of large language models for code

Original

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica · 2024

Earlier work this paper cites.

Numinamath

Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu · 2024

Cited alongside, same era.

Free process rewards without process labels

Original

Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng · 2024

Cited alongside, same era.

Tülu 3: Pushing frontiers in open language model post-training

Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, Yuling Gu, et al · 2024

Cited alongside, same era.

Infinity instruct

Beijing Academy of Artificial Intelligence (BAAI) · 2024

Cited alongside, same era.

Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation, 2024

Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu · 2024

Cited alongside, same era.

Am-thinking-v1: Advancing the frontier of reasoning at 32b scale, 2025

Yunjie Ji, Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yiping Peng, Han Zhao, and Xiangang Li · 2025

Closest in time.

Aime-preview: A rigorous and immediate evaluation framework for advanced mathematical reasoning

Yixin Ye, Yang Xiao, Tiantian Mi, and Pengfei Liu · 2025

Closest in time.

Open r1: A fully open reproduction of deepseek-r1, January 2025

Hugging Face · 2025

Closest in time.

Big-math: A large-scale, high-quality math dataset for reinforcement learning in language models, 2025

Alon Albalak, Duy Phung, Nathan Lile, Rafael Rafailov, Kanishk Gandhi, Louis Castricato, Anikait Singh, Chase Blagden, Violet Xiang, Dakota Mahan, and Nick Haber · 2025

Closest in time.

Deepcoder: A fully open-source 14b coder at o3-mini level

Michael Luo, Sijun Tan, Roy Huang, Ameen Patel, Alpay Ariyak, Qingyang Wu, Xiaoxiang Shi, Rachel Xin, Colin Cai, Maurice Weber, Ce Zhang, Li Erran Li, Raluca Ada Popa, and Ion Stoica · 2025

Closest in time.