2019

Branched Multi-Task Networks: Deciding What Layers To Share

Vandenhende, Simon, Georgoulis, Stamatios, De Brabandere, Bert et al.

Understand

In the context of multi-task learning, neural networks with branched architectures have often been employed to jointly tackle the tasks at hand.

Such ramified networks typically start with a number of shared layers, after which different tasks branch out into their own sequence of layers.
Understandably, as the number of possible network configurations is combinatorially large, deciding what layers to share and where to branch out becomes cumbersome.
Prior works have either relied on ad hoc methods to determine the level of layer sharing, which is suboptimal, or utilized neural architecture search techniques to establish the network design, which is considerably expensive.

Built on

Multitask learning
Rich Caruana · 1997
Earlier work this paper cites.
Regularized multi–task learning
Theodoros Evgeniou and Massimiliano Pontil · 2004
Earlier work this paper cites.
Model selection and estimation in regression with grouped variables
Ming Yuan and Yi Lin · 2006
Earlier work this paper cites.
Multi-task feature learning
Andreas Argyriou, Theodoros Evgeniou, and Massimiliano Pontil · 2007
Earlier work this paper cites.
Direction selectivity in v1 of alert monkeys: evidence for parallel pathways for motion processing
Moshe Gur and D Max Snodderly · 2007
Earlier work this paper cites.
A new approach to collaborative filtering: Operator estimation with spectral regularization
Jacob Abernethy, Francis Bach, Theodoros Evgeniou, and Jean-Philippe Vert · 2009
Earlier work this paper cites.
Taking advantage of sparsity in multi-task learning
Karim Lounici, Massimiliano Pontil, Alexandre B Tsybakov, and Sara Van De Geer · 2009
Earlier work this paper cites.
Learning multiple tasks using manifold regularization
Arvind Agarwal, Samuel Gerber, and Hal Daume · 2010
Earlier work this paper cites.
A dirty model for multi-task learning
Ali Jalali, Sujay Sanghavi, Chao Ruan, and Pradeep K Ravikumar · 2010
Earlier work this paper cites.
A survey on transfer learning
Sinno Jialin Pan, Qiang Yang, et al · 2010
Earlier work this paper cites.
Malsar: Multi-task learning via structural regularization
Jiayu Zhou, Jianhui Chen, and Jieping Ye · 2011
Earlier work this paper cites.
Learning task grouping and overlap in multi-task learning
Abhishek Kumar and Hal Daume III · 2012
Earlier work this paper cites.
Spatial pyramid pooling in deep convolutional networks for visual recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2015
Earlier work this paper cites.
Deep learning face attributes in the wild
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang · 2015
Earlier work this paper cites.
The cityscapes dataset for semantic urban scene understanding
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele · 2016
Earlier work this paper cites.
Cross-stitch networks for multi-task learning
Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Martial Hebert · 2016
Earlier work this paper cites.

Similar

Moon: A mixed objective optimization network for the recognition of facial attributes
Ethan M Rudd, Manuel Günther, and Terrance E Boult · 2016
Cited alongside, same era.
Attributes for improved attributes: A multi-task network utilizing implicit and explicit relationships for facial attribute classification
Emily M Hand and Rama Chellappa · 2017
Cited alongside, same era.
Ubernet: Training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory
Iasonas Kokkinos · 2017
Cited alongside, same era.
Distributed multi-task relationship learning
Sulin Liu, Sinno Jialin Pan, and Qirong Ho · 2017
Cited alongside, same era.
Learning multiple tasks with multilinear relationship networks
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and S Yu Philip · 2017
Cited alongside, same era.
Fully-adaptive feature sharing in multi-task networks with applications in person attribute classification
Yongxi Lu, Abhishek Kumar, Shuangfei Zhai, Yu Cheng, Tara Javidi, and Rogerio Feris · 2017
Cited alongside, same era.
Fast scene understanding for autonomous driving
Davy Neven, Bert De Brabandere, Stamatios Georgoulis, Marc Proesmans, and Luc Van Gool · 2017
Cited alongside, same era.
An overview of multi-task learning in deep neural networks
Original
Sebastian Ruder · 2017
Cited alongside, same era.
Neural architecture search with reinforcement learning
Barret Zoph and Quoc V Le · 2017
Cited alongside, same era.
Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks
Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich · 2018
Cited alongside, same era.
Large scale fine-grained categorization and domain-specific transfer learning
Yin Cui, Yang Song, Chen Sun, Andrew Howard, and Serge Belongie · 2018
Cited alongside, same era.
Dynamic task prioritization for multitask learning
Michelle Guo, Albert Haque, De-An Huang, Serena Yeung, and Li Fei-Fei · 2018
Cited alongside, same era.

Then

Efficient neural architecture search via parameter sharing
Hieu Pham, Melody Y Guan, Barret Zoph, Quoc V Le, and Jeff Dean · 2018
Later among the works it cites.
Routing networks: Adaptive selection of non-linear functions for multi-task learning
Clemens Rosenbaum, Tim Klinger, and Matthew Riemer · 2018
Later among the works it cites.
Multi-task learning as multi-objective optimization
Ozan Sener and Vladlen Koltun · 2018
Later among the works it cites.
Pad-net: Multi-tasks guided prediction-and-distillation network for simultaneous depth estimation and scene parsing
Dan Xu, Wanli Ouyang, Xiaogang Wang, and Nicu Sebe · 2018
Later among the works it cites.
Taskonomy: Disentangling task transfer learning
Amir R Zamir, Alexander Sax, William Shen, Leonidas J Guibas, Jitendra Malik, and Silvio Savarese · 2018
Later among the works it cites.
A modulation module for multi-task learning with applications in image retrieval
Xiangyun Zhao, Haoxiang Li, Xiaohui Shen, Xiaodan Liang, and Ying Wu · 2018
Later among the works it cites.
Task2vec: Task embedding for meta-learning
Alessandro Achille, Michael Lam, Rahul Tewari, Avinash Ravichandran, Subhransu Maji, Charless C Fowlkes, Stefano Soatto, and Pietro Perona · 2019
Closest in time.
Stochastic filter groups for multi-task cnns: Learning specialist and generalist convolution kernels
Felix JS Bragman, Ryutaro Tanno, Sebastien Ourselin, Daniel C Alexander, and M Jorge Cardoso · 2019
Closest in time.
Representation similarity analysis for efficient task taxonomy & transfer learning
Kshitij Dwivedi and Gemma Roig · 2019
Closest in time.
Neural architecture search: A survey
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter · 2019
Closest in time.
Nddr-cnn: Layerwise feature fusing in multi-task cnns by neural discriminative dimensionality reduction
Yuan Gao, Jiayi Ma, Mingbo Zhao, Wei Liu, and Alan L Yuille · 2019
Closest in time.
End-to-end multi-task learning with attention
Shikun Liu, Edward Johns, and Andrew J Davison · 2019
Closest in time.
Attentive single-tasking of multiple tasks
Kevis-Kokitsi Maninis, Ilija Radosavovic, and Iasonas Kokkinos · 2019
Closest in time.
Feature partitioning for efficient multi-task architectures
Original
Alejandro Newell, Lu Jiang, Chong Wang, Li-Jia Li, and Jia Deng · 2019
Closest in time.
Regularized evolution for image classifier architecture search
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…