Task Singular Vectors: Reducing Task Interference in Model Merging

The approximation of one matrix by another of lower rank

Carl Eckart and G. Marion Young · 1936

Earlier work this paper cites.

An Analysis of Single-Layer Networks in Unsupervised Feature Learning

Adam Coates, Andrew Ng, and Honglak Lee · 1938

Earlier work this paper cites.

Note on a method for calculating corrected sums of squares and products

B. P. Welford · 1962

Earlier work this paper cites.

Gradient-based learning applied to document recognition

Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner · 1998

Earlier work this paper cites.

Automated Flower Classification over a Large Number of Classes

Maria-Elena Nilsback and Andrew Zisserman · 2008

Earlier work this paper cites.

Tensor decompositions and applications

Tamara G Kolda and Brett W Bader · 2009

Earlier work this paper cites.

Learning multiple layers of features from tiny images

Alex Krizhevsky and Geoffrey Hinton · 2009

Earlier work this paper cites.

Reading digits in natural images with unsupervised feature learning

Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng · 2011

Earlier work this paper cites.

The German Traffic Sign Recognition Benchmark: A multi-class classification competition

Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel · 2011

Earlier work this paper cites.

Cats and dogs

Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar · 2012

Earlier work this paper cites.

Predicting parameters in deep learning

Misha Denil, Babak Shakibi, Laurent Dinh, Marc’Aurelio Ranzato, and Nando De Freitas · 2013

Earlier work this paper cites.

Challenges in Representation Learning: A Report on Three Machine Learning Contests

Ian J. Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, Yingbo Zhou, Chetan Ramaiah, Fangxiang Feng, Ruifan Li, Xiaojie Wang, Dimitris Athanasakis, John Shawe-Taylor, Maxim Milakov, John Park, Radu Ionescu, Marius Popescu, Cristian Grozea, James Bergstra, Jingjing Xie, Lukasz Romaszko, Bing Xu, Zhang Chuang, and Yoshua Bengio · 2013

Earlier work this paper cites.

3D Object Representations for Fine-Grained Categorization

Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei · 2013

Earlier work this paper cites.

Recursive deep models for semantic compositionality over a sentiment treebank

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts · 2013

Earlier work this paper cites.

Food-101 – Mining Discriminative Components with Random Forests

Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool · 2014

Earlier work this paper cites.

Describing Textures in the Wild

Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi · 2014

Earlier work this paper cites.

Exploiting linear structure within convolutional networks for efficient evaluation

Emily L Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus · 2014

Earlier work this paper cites.

Learning factored representations in a deep mixture of experts

David Eigen, Marc’Aurelio Ranzato, and Ilya Sutskever · 2014

Earlier work this paper cites.

The optimal hard threshold for singular values is

Matan Gavish and David L. Donoho · 2014

Cited alongside, same era.

Speeding up convolutional neural networks with low rank expansions

Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman · 2014

Cited alongside, same era.

Ultimate tensorization: compressing convolutional and fc layers alike

Original

Timur Garipov, Dmitry Podoprikhin, Alexander Novikov, and Dmitry Vetrov · 2016

Cited alongside, same era.

Compression of deep convolutional neural networks for fast and low power mobile applications

Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, and Dongjun Shin · 2016

Cited alongside, same era.

SUN Database: Exploring a Large Collection of Scene Categories

Jianxiong Xiao, Krista A. Ehinger, James Hays, Antonio Torralba, and Aude Oliva · 2016

Cited alongside, same era.

Git re-basin: Merging models modulo permutation symmetries

Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa · 2023

Later among the works it cites.

Editing models with task arithmetic

Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi · 2023

Later among the works it cites.

Dataless knowledge fusion by merging weights of language models

Xisen Jin, Xiang Ren, Daniel Preotiuc-Pietro, and Pengxiang Cheng · 2023

Later among the works it cites.

REPAIR: REnormalizing permuted activations for interpolation repair

Keller Jordan, Hanie Sedghi, Olga Saukh, Rahim Entezari, and Behnam Neyshabur · 2023

Later among the works it cites.

Re-basin via implicit sinkhorn differentiation

Fidel A Guerrero Peña, Heitor Rapela Medeiros, Thomas Dubail, Masih Aminbeidokhti, Eric Granger, and Marco Pedersoli · 2023

Later among the works it cites.

Model merging by uncertainty-based gradient matching

Nico Daheim, Thomas Möllenhoff, Edoardo Ponti, Iryna Gurevych, and Mohammad Emtiyaz Khan · 2024