Fetching the paper…
Reading the bibliography…
It is common within the deep learning community to first pre-train a deep neural network from a large-scale dataset and then fine-tune the pre-trained model to a specific downstream task.
Self-organizing neural network that discovers surfaces in random-dot stereograms
Becker, S. and Hinton, G. E · 1992
Earlier work this paper cites.
Dimensionality reduction by learning an invariant mapping
Hadsell, R., Chopra, S., and Lecun, Y · 2006
Earlier work this paper cites.
Visualizing data using t-sne
Maaten, L. v. d. and Hinton, G · 2008
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
Gutmann, M. and Hyvärinen, A · 2010
Earlier work this paper cites.
Caltech-UCSD Birds 200
Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P · 2010
Earlier work this paper cites.
Cats and dogs
Parkhi, O. M., Vedaldi, A., Zisserman, A., and Jawahar, C · 2012
Earlier work this paper cites.
Colorful image colorization
Zhang, R., Isola, P., and Efros, A. A · 2012
Earlier work this paper cites.
3d object representations for fine-grained categorization
Krause, J., Stark, M., Deng, J., and Fei-Fei, L · 2013
Earlier work this paper cites.
Fine-grained visual classification of aircraft
Maji, S., Rahtu, E., Kannala, J., Blaschko, M., and Vedaldi, A · 2013
Earlier work this paper cites.
Generative adversarial nets
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y · 2014
Earlier work this paper cites.
Microsoft coco: Common objects in context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L · 2014
Earlier work this paper cites.
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., and Lipson, H · 2014
Earlier work this paper cites.
Unsupervised visual representation learning by context prediction
Doersch, C., Gupta, A., and Efros, A. A · 2015
Cited alongside, same era.
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S. E., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A · 2015
Cited alongside, same era.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Cited alongside, same era.
Unsupervised learning of visual representations by solving jigsaw puzzles
Noroozi, M. and Favaro, P · 2016
Cited alongside, same era.
A discriminative feature learning approach for deep face recognition
Wen, Y., Zhang, K., Li, Z., and Qiao, Y · 2016
Cited alongside, same era.
Multi-task self-supervised visual learning
Doersch, C. and Zisserman, A · 2017
Cited alongside, same era.
Representation learning with contrastive predictive coding, 2018
van den Oord, A., Li, Y., and Vinyals, O · 2018
Later among the works it cites.
Unsupervised feature learning via non-parametric instance discrimination
Wu, Z., Xiong, Y., Yu, S. X., and Lin, D · 2018
Later among the works it cites.
Catastrophic forgetting meets negative transfer: Batch spectral shrinkage for safe transfer learning
Chen, X., Wang, S., Fu, B., Long, M., and Wang, J · 2019
Later among the works it cites.
Rethinking imagenet pre-training
He, K., Girshick, R. B., and Dollár, P · 2019
Later among the works it cites.
Structure learning with similarity preserving
Kang, Z., Lu, X., Lu, Y., Peng, C., and Xu, Z · 2019
Later among the works it cites.
Do better imagenet models transfer better?
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Interpretable explanations of black boxes by meaningful perturbation
Fong, R. C. and Vedaldi, A · 2017
Cited alongside, same era.
Densely connected convolutional networks
Huang, G., Liu, Z., van der Maaten, L., and Weinberger, K. Q · 2017
Cited alongside, same era.
Learning without forgetting
Li, Z. and Hoiem, D · 2017
Cited alongside, same era.
Deep clustering for unsupervised learning of visual features
Caron, M., Bojanowski, P., Joulin, A., and Douze, M · 2018
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Cited alongside, same era.
Explicit inductive bias for transfer learning with convolutional networks
Li, X., Grandvalet, Y., and Davoine, F · 2018
Cited alongside, same era.
Kornblith, S., Shlens, J., and Le, Q. V · 2019
Later among the works it cites.
Delta: Deep learning transfer using feature map with attention for convolutional networks
Li, X., Xiong, H., Wang, H., Rao, Y., Liu, L., and Huan, J · 2019
Later among the works it cites.
Language models are unsupervised multitask learners
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I · 2019
Later among the works it cites.
Tian, Y., Krishnan, D., and Isola, P · 2019
Later among the works it cites.
Xlnet: Generalized autoregressive pretraining for language understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J. G., Salakhutdinov, R., and Le, Q. V · 2019
Later among the works it cites.
A Simple Framework for Contrastive Learning of Visual Representations
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G · 2020
Closest in time.
Momentum contrast for unsupervised visual representation learning
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R · 2020
Closest in time.