Knowledge Distillation for Multi-task Learning

Caruana, R.: Multitask learning. Machine learning 28

1997

Earlier work this paper cites.

Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: Advances in Neural Information Processing Systems Workshop on Deep Learning and Unsupervised Feature Learning (2011)

2011

Earlier work this paper cites.

Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: European conference on computer vision. pp. 746–760. Springer (2012)

2012

Earlier work this paper cites.

Seltzer, M.L., Droppo, J.: Multi-task learning in deep neural networks for improved phoneme recognition. In: IEEE International Conference on Acoustics, Speech and Signal Processing. pp. 6965–6969. IEEE (2013)

2013

Earlier work this paper cites.

Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

Original

2014

Earlier work this paper cites.

Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: Fitnets: Hints for thin deep nets. arXiv preprint arXiv:1412.6550 (2014)

Original

2014

Earlier work this paper cites.

Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Facial landmark detection by deep multi-task learning. In: European conference on computer vision. pp. 94–108. Springer (2014)

2014

Earlier work this paper cites.

Eigen, D., Fergus, R.: Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture. In: IEEE International Conference on Computer Vision. pp. 2650–2658 (2015)

2015

Earlier work this paper cites.

Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)

Original

2015

Earlier work this paper cites.

Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B.: Human-level concept learning through probabilistic program induction. Science 350

2015

Earlier work this paper cites.

Parisotto, E., Ba, J.L., Salakhutdinov, R.: Actor-mimic: Deep multitask and transfer reinforcement learning. arXiv preprint arXiv:1511.06342 (2015)

Original

2015

Earlier work this paper cites.

Bilen, H., Vedaldi, A.: Integrated perception with recurrent multi-task neural networks. In: Advances in Neural Information Processing Systems. pp. 235–243 (2016)

2016

Cited alongside, same era.

Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., Schiele, B.: The cityscapes dataset for semantic urban scene understanding. In: Computer Vision and Pattern Recognition. pp. 3213–3223 (2016)

2016

Cited alongside, same era.

Misra, I., Shrivastava, A., Gupta, A., Hebert, M.: Cross-stitch networks for multi-task learning. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 3994–4003 (2016)

2016

Cited alongside, same era.

Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation. Transactions on Pattern Analysis and Machine Intelligence 39

2017

Cited alongside, same era.

Kendall, A., Gal, Y., Cipolla, R.: Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 7482–7491 (2018)

2018

Later among the works it cites.

Meyerson, E., Miikkulainen, R.: Pseudo-task augmentation: From deep multitask learning to intratask sharing—and back. arXiv preprint arXiv:1803.04062 (2018)

Original

2018

Later among the works it cites.

Rebuffi, S.A., Bilen, H., Vedaldi, A.: Efficient parametrization of multi-domain deep neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 8119–8127 (2018)

2018

Later among the works it cites.

Sener, O., Koltun, V.: Multi-task learning as multi-objective optimization. In: Advances in Neural Information Processing Systems. pp. 527–538 (2018)

2018

Later among the works it cites.