Revisiting Model Interpolation for Efficient Reasoning

Weight averaging for neural networks and local resampling schemes

Joachim Utans. 1996 · 1996

Earlier work this paper cites.

A comprehensive survey on model compression and acceleration

Tejalal Choudhary, Vipul Mishra, Anurag Goswami, and Jagannathan Sarangapani. 2020 · 2020

Earlier work this paper cites.

Evaluating large language models trained on code

Original

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and 1 others. 2021 · 2021

Earlier work this paper cites.

Editing models with task arithmetic

Original

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. 2022 · 2022

Earlier work this paper cites.

Chain-of-thought prompting elicits reasoning in large language models

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022 · 2022

Earlier work this paper cites.

Opencompass: A universal evaluation platform for foundation models

OpenCompass Contributors. 2023 · 2023

Earlier work this paper cites.

Soft merging of experts with adaptive routing

Original

Mohammed Muqeeth, Haokun Liu, and Colin Raffel. 2023 · 2023

Earlier work this paper cites.

Resolving interference when merging models

Original

Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal. 2023 · 2023

Earlier work this paper cites.

Adamerging: Adaptive model merging for multi-task learning

Original

Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao. 2023 · 2023

Earlier work this paper cites.

Language models are super mario: Absorbing abilities from homologous models as a free lunch

Original

Le Yu, Yu Bowen, Haiyang Yu, Fei Huang, and Yongbin Li. 2023 · 2023

Earlier work this paper cites.

Instruction-following evaluation for large language models

Original

Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023 · 2023

Earlier work this paper cites.

Aime problems and solutions

AIME. 2025 · 2024

Earlier work this paper cites.

Do not think that much for 2+ 3=? on the overthinking of o1-like llms

Original

Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, and 1 others. 2024 · 2024

Cited alongside, same era.

Arcee’s mergekit: A toolkit for merging large language models

Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. 2024 · 2024

Cited alongside, same era.

Training large language models to reason in a continuous latent space

Original

Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2024 · 2024

Cited alongside, same era.

Openai o1 system card

Original

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024 · 2024

Cited alongside, same era.

Sketch-of-thought: Efficient llm reasoning with adaptive cognitive-inspired sketching

Original

Simon A Aytes, Jinheon Baek, and Sung Ju Hwang. 2025 · 2025

Closest in time.

Soft tokens, hard truths

Original

Natasha Butt, Ariel Kwiatkowski, Ismail Labiad, Julia Kempe, and Yann Ollivier. 2025 · 2025

Closest in time.

Merge now, regret later: The hidden cost of model merging is adversarial transferability

Original

Ankit Gangwal and Aaryan Ajay Sharma. 2025 · 2025

Closest in time.

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

Original

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025 · 2025

Closest in time.

Tl; dr: Too long, do re-weighting for effcient llm reasoning compression