ATM: Improving Model Merging by Alternating Tuning and Merging

Gradient-based learning applied to document recognition

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner · 1998

Earlier work this paper cites.

Sun database: Large-scale scene recognition from abbey to zoo

Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba · 2010

Earlier work this paper cites.

Reading digits in natural images with unsupervised feature learning

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Baolin Wu, Andrew Y Ng, et al · 2011

Earlier work this paper cites.

Detection of traffic signs in real-world images: The German Traffic Sign Detection Benchmark

Sebastian Houben, Johannes Stallkamp, Jan Salmen, Marc Schlipsing, and Christian Igel · 2013

Earlier work this paper cites.

Describing textures in the wild

Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi · 2014

Earlier work this paper cites.

Remote sensing image scene classification: Benchmark and state of the art

Gong Cheng, Junwei Han, and Xiaoqiang Lu · 2017

Earlier work this paper cites.

Communication-efficient learning of deep networks from decentralized data

Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas · 2017

Earlier work this paper cites.

Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth · 2019

Earlier work this paper cites.

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby · 2021

Earlier work this paper cites.

Merging models with fisher-weighted averaging

Michael Matena and Colin Raffel · 2021

Cited alongside, same era.

Fusing finetuned models for better pretraining

Original

Leshem Choshen, Elad Venezian, Noam Slonim, and Yoav Katz · 2022

Cited alongside, same era.

Editing models with task arithmetic

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi · 2022

Cited alongside, same era.

Dataless knowledge fusion by merging weights of language models

Original

Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng · 2022

Cited alongside, same era.

Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Mitchell Wortsman, Gabriel Ilharco, Samir Yitzhak Gadre, Rebecca Roelofs, Raphael Gontijo Lopes, Ari S. Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, and Ludwig Schmidt · 2022

Equivariant deep weight space alignment, 2023

Aviv Navon, Aviv Shamsian, Ethan Fetaya, Gal Chechik, Nadav Dym, and Haggai Maron · 2023

Later among the works it cites.

Ties-merging: Resolving interference when merging models

Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal · 2023

Later among the works it cites.

Adamerging: Adaptive model merging for multi-task learning

Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao · 2023

Later among the works it cites.

Language models are super mario: Absorbing abilities from homologous models as a free lunch

Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li · 2023

Later among the works it cites.

c 2 ​ m 3 c^{2}m^{3} : Cycle-consistent multi-model merging

Donato Crisostomi, Marco Fumero, Daniele Baieri, Florian Bernard, and Emanuele Rodolà · 2024

Closest in time.

Task arithmetic in the tangent spacekrizhevsky2009learning: Improved editing of pre-trained models