Built on

K. Clark, M. Luong, U. Khandelwal, C. D. Manning, and Q. V. Le, “Bam! born-again multi-task networks for natural language understanding,” CoRR , vol. abs/1907.04829, 2019. [Online]. Available: http://arxiv.org/abs/1907.04829
Original
1907
Earlier work this paper cites.
Y. Sun, S. Wang, Y. Li, S. Feng, H. Tian, H. Wu, and H. Wang, “ERNIE 2.0: A continual pre-training framework for language understanding,” CoRR , vol. abs/1907.12412, 2019. [Online]. Available: http://arxiv.org/abs/1907.12412
Original
1907
Earlier work this paper cites.
R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” in Machine Learning , vol. 8(3–4), 1992, pp. 229––256
1992
Earlier work this paper cites.
——, “Multitask learning: A knowledge-based source of inductive bias,” In Machine Learning: Proceedings of the Tenth International Conference , pp. 41–48, 1993
1993
Earlier work this paper cites.
R. Caruana, “Multitask learning,” Learning to learn , 1998
1998
Earlier work this paper cites.
A. Krizhevsky et al. , “Learning multiple layers of features from tiny images,” 2009
2009
Earlier work this paper cites.
R. Girshick, “Fast R-CNN,” in International Conference on Computer Vision (ICCV) . IEEE, 2015, pp. 1440––1448
2015
Earlier work this paper cites.
M. B. Lake, R. Salakhutdinov, and J. B. Tenenbaum, “Human-level concept learning through probabilistic program induction,” Science , vol. 350, p. 1332–1338, 2015
2015
Earlier work this paper cites.
T. Bansal, D. Belanger, and A. McCallum, “Ask the GRU: Multi-task learning for deep text recommendations,” in 10th ACM Conference on Recommender Systems . ACM, 2016, pp. 107––114
2016
Earlier work this paper cites.
I. Misra, A. Shrivastava, A. Gupta, , and M. Hebert, “Cross-stitch networks for multi-task learning,” in Conference on Computer Vision and Pattern Recognition . IEEE, 2016, pp. 3994––4003
2016
Earlier work this paper cites.
E. Bengio, P.-L. Bacon, J. Pineau, and D. Precup, “Conditional computation in neural networks for faster models,” arXiv preprint arXiv:1511.06297 , 2016
Original
2016
Earlier work this paper cites.

Similar

E. Jang, S. Gu, and B. Poole, “Categorical reparametrization with Gumbel-softmax,” International Conference on Learning Representations (ICLR) , 2017
2017
Cited alongside, same era.
N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” arXiv preprint arXiv:1701.06538 , 2017
Original
2017
Cited alongside, same era.
C. Fernando, D. Banarse, C. Blundell, Y. Zwols, D. Ha, A. A. Rusu, A. Pritzel, and D. Wierstra, “Pathnet: Evolution channels gradient descent in super neural networks,” arXiv preprint arXiv:1701.08734 , 2017
Original
2017
Cited alongside, same era.
B. Zoph and Q. V. Le, “Neural Architecture Search with Reinforcement Learning,” International Conference on Learning Representations (ICLR) , 2017
2017
Cited alongside, same era.

Then

H. Pham, M. Guan, B. Zoph, Q. Le, and J. Dean, “Efficient neural architecture search via parameters sharing,” International Conference on Machine Learning (ICML) , 2018
2018
Later among the works it cites.
J. Liang, E. Meyerson, and R. Miikkulainen, “Evolutionary architecture search for deep multitask networks,” in Proceedings of the Genetic and Evolutionary Computation Conference . ACM, 2018, pp. 466–473
2018
Later among the works it cites.
Y. Wu and K. He, “Group normalization,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 3–19
2018
Later among the works it cites.
C. Rosenbaum, I. Cases, M. Riemer, and T. Klinger, “Routing networks and the challenges of modular and compositional computation,” arXiv preprint arXiv:1904.12774 , 2019
Original
2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2019

Flexible Multi-task Networks by Learning Parameter Allocation

Maziarz, Krzysztof, Kokiopoulou, Efi, Gesmundo, Andrea et al.

Understand

This paper proposes a novel learning method for multi-task applications.

Multi-task neural networks can learn to transfer knowledge across different tasks by using parameter sharing.
However, sharing parameters between unrelated tasks can hurt performance.
To address this issue, we propose a framework to learn fine-grained patterns of parameter sharing.

Built on

K. Clark, M. Luong, U. Khandelwal, C. D. Manning, and Q. V. Le, “Bam! born-again multi-task networks for natural language understanding,” CoRR , vol. abs/1907.04829, 2019. [Online]. Available: http://arxiv.org/abs/1907.04829
Original
1907
Earlier work this paper cites.
Y. Sun, S. Wang, Y. Li, S. Feng, H. Tian, H. Wu, and H. Wang, “ERNIE 2.0: A continual pre-training framework for language understanding,” CoRR , vol. abs/1907.12412, 2019. [Online]. Available: http://arxiv.org/abs/1907.12412
Original
1907
Earlier work this paper cites.
R. J. Williams, “Simple statistical gradient-following algorithms for connectionist reinforcement learning,” in Machine Learning , vol. 8(3–4), 1992, pp. 229––256
1992
Earlier work this paper cites.
——, “Multitask learning: A knowledge-based source of inductive bias,” In Machine Learning: Proceedings of the Tenth International Conference , pp. 41–48, 1993
1993
Earlier work this paper cites.
R. Caruana, “Multitask learning,” Learning to learn , 1998
1998
Earlier work this paper cites.
A. Krizhevsky et al. , “Learning multiple layers of features from tiny images,” 2009
2009
Earlier work this paper cites.
R. Girshick, “Fast R-CNN,” in International Conference on Computer Vision (ICCV) . IEEE, 2015, pp. 1440––1448
2015
Earlier work this paper cites.
M. B. Lake, R. Salakhutdinov, and J. B. Tenenbaum, “Human-level concept learning through probabilistic program induction,” Science , vol. 350, p. 1332–1338, 2015
2015
Earlier work this paper cites.
T. Bansal, D. Belanger, and A. McCallum, “Ask the GRU: Multi-task learning for deep text recommendations,” in 10th ACM Conference on Recommender Systems . ACM, 2016, pp. 107––114
2016
Earlier work this paper cites.
I. Misra, A. Shrivastava, A. Gupta, , and M. Hebert, “Cross-stitch networks for multi-task learning,” in Conference on Computer Vision and Pattern Recognition . IEEE, 2016, pp. 3994––4003
2016
Earlier work this paper cites.
E. Bengio, P.-L. Bacon, J. Pineau, and D. Precup, “Conditional computation in neural networks for faster models,” arXiv preprint arXiv:1511.06297 , 2016
Original
2016
Earlier work this paper cites.

Similar

E. Jang, S. Gu, and B. Poole, “Categorical reparametrization with Gumbel-softmax,” International Conference on Learning Representations (ICLR) , 2017
2017
Cited alongside, same era.
N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously large neural networks: The sparsely-gated mixture-of-experts layer,” arXiv preprint arXiv:1701.06538 , 2017
Original
2017
Cited alongside, same era.
C. Fernando, D. Banarse, C. Blundell, Y. Zwols, D. Ha, A. A. Rusu, A. Pritzel, and D. Wierstra, “Pathnet: Evolution channels gradient descent in super neural networks,” arXiv preprint arXiv:1701.08734 , 2017
Original
2017
Cited alongside, same era.
B. Zoph and Q. V. Le, “Neural Architecture Search with Reinforcement Learning,” International Conference on Learning Representations (ICLR) , 2017
2017
Cited alongside, same era.

Then

H. Pham, M. Guan, B. Zoph, Q. Le, and J. Dean, “Efficient neural architecture search via parameters sharing,” International Conference on Machine Learning (ICML) , 2018
2018
Later among the works it cites.
J. Liang, E. Meyerson, and R. Miikkulainen, “Evolutionary architecture search for deep multitask networks,” in Proceedings of the Genetic and Evolutionary Computation Conference . ACM, 2018, pp. 466–473
2018
Later among the works it cites.
Y. Wu and K. He, “Group normalization,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2018, pp. 3–19
2018
Later among the works it cites.
C. Rosenbaum, I. Cases, M. Riemer, and T. Klinger, “Routing networks and the challenges of modular and compositional computation,” arXiv preprint arXiv:1904.12774 , 2019
Original
2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

E. Real, S. Moore, A. Selle, S. Saxena, Y. L. Suematsu, J. Tan, Q. V. Le, and A. Kurakin, “Large-Scale Evolution of Image Classifiers,” International Conference on Machine Learning (ICML) , 2017

2017

Cited alongside, same era.

H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747 , 2017

Original

2017

Cited alongside, same era.

J. Ma, Z. Zhao, Y. X., J. Chen, L. Hong, and E. Chi, “Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts,” KDD , 2018

2018

Cited alongside, same era.

Y. Guo, H. Shi, A. Kumar, K. Grauman, T. Rosing, and R. Feris, “SpotTune: Transfer Learning through Adaptive Fine-tuning,” Conference on Computer Vision and Pattern Recognition (CVPR) , 2018. [Online]. Available: https://arxiv.org/abs/1811.08737

Original

2018

Cited alongside, same era.

C. Rosenbaum, T. Klinger, and M. Riemer, “Routing Networks: Adaptive Selection of Non-Linear Functions for Multi-Task Learning,” International Conference on Learning Representations (ICLR) , 2018. [Online]. Available: https://openreview.net/pdf?id=ry8dvM-R-

2018

Cited alongside, same era.

E. Meyerson and R. Miikkulainen, “Beyond shared hierarchies: Deep multitask learning through soft layer ordering,” International Conference on Learning Representations (ICLR) , 2018

2018

Cited alongside, same era.

K. Maziarz, M. Tan, A. Khorlin, M. Georgiev, and A. Gesmundo, “Evolutionary-neural hybrid agents for architecture search,” International Conference on Machine Learning (ICML): Workshop on AutoML , 2018

2018

Cited alongside, same era.

P. Ramachandran and Q. V. Le, “Diversity and Depth in Per-Example Routing Models,” International Conference on Learning Representations (ICLR) , 2019. [Online]. Available: https://openreview.net/pdf?id=BkxWJnC9tX

2019

Closest in time.

J. Ma, Z. Zhao, J. Chen, A. Li, L. Hong, and E. Chi, “SNR: Sub-Network Routing for Flexible Parameter Sharing in Multi-task Learning,” AAAI Conference on Artificial Intelligence , 2019. [Online]. Available: http://www.jiaqima.com/papers/SNR.pdf

2019

Closest in time.

G. Strezoski, N. van Noord, and M. Worring, “Many Task Learning With Task Routing,” in Proceedings of the IEEE International Conference on Computer Vision , 2019, pp. 1375–1384

2019

Closest in time.

A. Veit and S. Belongie, “Convolutional Networks with Adaptive Inference Graphs,” in International Journal of Computer Vision (IJCV) , 2019, pp. 1––12

2019

Closest in time.

C. White, W. Neiswanger, and Y. Savani, “Bananas: Bayesian optimization with neural architectures for neural architecture search,” arXiv preprint arXiv:1910.11858 , 2019

Original

2019

Closest in time.

H. Liu, K. Simonyan, and Y. Yang, “DARTS: Differentiable architecture search,” in International Conference on Learning Representations (ICLR) , 2019

2019

Closest in time.