Fetching the paper…
Reading the bibliography…
Fusion is a technique for merging multiple independently-trained neural networks in order to combine their capabilities.
Neural networks and the bias/variance dilemma
Stuart Geman, Elie Bienenstock, and René Doursat · 1992
Earlier work this paper cites.
Bagging predictors
Leo Breiman · 1996
Earlier work this paper cites.
The earth mover’s distance as a metric for image retrieval
Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas · 2000
Earlier work this paper cites.
The sinkhorn–knopp algorithm: convergence and applications
Philip A Knight · 2008
Earlier work this paper cites.
Optimal transport: old and new , volume 338
Cédric Villani et al · 2009
Earlier work this paper cites.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby · 2010
Earlier work this paper cites.
Sinkhorn distances: Lightspeed computation of optimal transport
Marco Cuturi · 2013
Earlier work this paper cites.
Wasserstein gan, 2017
Martin Arjovsky, Soumith Chintala, and Léon Bottou · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
BERT: pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova · 2018
Earlier work this paper cites.
Loss surfaces, mode connectivity, and fast ensembling of dnns
Timur Garipov, Pavel Izmailov, Dmitrii Podoprikhin, Dmitry P Vetrov, and Andrew G Wilson · 2018
Earlier work this paper cites.
GLUE: A multi-task benchmark and analysis platform for natural language understanding
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman · 2018
Earlier work this paper cites.
Scaling laws for neural language models
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei · 2020
Cited alongside, same era.
Model fusion via optimal transport
Sidak Pal Singh and Martin Jaggi · 2020
Cited alongside, same era.
Federated learning with matched averaging
Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dimitris Papailiopoulos, and Yasaman Khazaeni · 2020
Cited alongside, same era.
On the opportunities and risks of foundation models
Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al · 2021
Cited alongside, same era.
The role of permutation invariance in linear mode connectivity of neural networks
A survey of transformers - sciencedirect
Tianyang Lin, Yuxin Wang, Xiangyang Liu, and Xipeng Qiu · 2022
Later among the works it cites.
Deep neural network fusion via graph matching with applications to model ensemble and federated learning
Chang Liu, Chenfei Lou, Runzhong Wang, Alan Yuhan Xi, Li Shen, and Junchi Yan · 2022
Later among the works it cites.
Merging models with fisher-weighted averaging
Michael S Matena and Colin A Raffel · 2022
Later among the works it cites.
A survey of ensemble learning: Concepts, algorithms, applications, and prospects
Ibomoiye Domor Mienye and Yanxia Sun · 2022
Later among the works it cites.
How do vision transformers work?, 2022
Namuk Park and Songkuk Kim · 2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Rahim Entezari, Hanie Sedghi, Olga Saukh, and Behnam Neyshabur · 2021
Cited alongside, same era.
Fusing multitask models by recursive least squares
Xiaobin Li, Lianlei Shan, and Weiqiang Wang · 2021
Cited alongside, same era.
Model fusion of heterogeneous neural networks via cross-layer alignment
Dang Nguyen, Khai Nguyen, Dinh Phung, Hung Bui, and Nhat Ho · 2021
Cited alongside, same era.
Co-transport for class-incremental learning
Da-Wei Zhou, Han-Jia Ye, and De-Chuan Zhan · 2021
Cited alongside, same era.
Git re-basin: Merging models modulo permutation symmetries
Samuel K Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa · 2022
Cited alongside, same era.
Wasserstein barycenter-based model fusion and linear mode connectivity of neural networks
Aditya Kumar Akash, Sixu Li, and Nicolás García Trillos · 2022
Cited alongside, same era.
On the symmetries of deep learning models and their internal representations
Charles Godfrey, Davis Brown, Tegan Emerson, and Henry Kvinge · 2022
Cited alongside, same era.
Training compute-optimal large language models
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al · 2022
Cited alongside, same era.
Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun · 2022
Later among the works it cites.
Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time
Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al · 2022
Later among the works it cites.
Scaling vision transformers
Xiaohua Zhai, Alexander Kolesnikov, Neil Houlsby, and Lucas Beyer · 2022
Later among the works it cites.
Scaling mlps: A tale of inductive bias
Gregor Bachmann, Sotiris Anagnostidis, and Thomas Hofmann · 2023
Closest in time.
Population parameter averaging (papa)
Alexia Jolicoeur-Martineau, Emy Gervais, Kilian Fatras, Yan Zhang, and Simon Lacoste-Julien · 2023
Closest in time.
On cross-layer alignment for model fusion of heterogeneous neural networks
Dang Nguyen, Trang Nguyen, Khai Nguyen, Dinh Phung, Hung Bui, and Nhat Ho · 2023
Closest in time.
Zipit! merging models from different tasks without training, 2023
George Stoica, Daniel Bolya, Jakob Bjorner, Taylor Hearn, and Judy Hoffman · 2023
Closest in time.