Fetching the paper…
Reading the bibliography…
Pre-training has been widely adopted in deep learning to improve model performance, especially when the training data for a target task is limited.
Dataset shift in machine learning
J. Quiñonero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence · 2008
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Evasion attacks against machine learning at test time
B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndić, P. Laskov, G. Giacinto, and F. Roli · 2013
Earlier work this paper cites.
Intriguing properties of neural networks
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus · 2013
Earlier work this paper cites.
Densenet: Implementing efficient convnet descriptor pyramids
F. Iandola, M. Moskewicz, S. Karayev, R. Girshick, T. Darrell, and K. Keutzer · 2014
Earlier work this paper cites.
Two-stream convolutional networks for action recognition in videos
K. Simonyan and A. Zisserman · 2014
Earlier work this paper cites.
Faster r-cnn: Towards real-time object detection with region proposal networks
S. Ren, K. He, R. Girshick, and J. Sun · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al · 2015
Earlier work this paper cites.
R-fcn: Object detection via region-based fully convolutional networks
J. Dai, Y. Li, K. He, and J. Sun · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
What makes imagenet good for transfer learning?
M. Huh, P. Agrawal, and A. A. Efros · 2016
Earlier work this paper cites.
Yfcc100m: The new data in multimedia research
B. Thomee, D. A. Shamma, G. Friedland, B. Elizalde, K. Ni, D. Poland, D. Borth, and L.-J. Li · 2016
Earlier work this paper cites.
Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille · 2017
Earlier work this paper cites.
Mask r-cnn
K. He, G. Gkioxari, P. Dollár, and R. Girshick · 2017
Earlier work this paper cites.
Speed/accuracy trade-offs for modern convolutional object detectors
J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, I. Fischer, Z. Wojna, Y. Song, S. Guadarrama, et al · 2017
Earlier work this paper cites.
Decoupled weight decay regularization
I. Loshchilov and F. Hutter · 2017
Cited alongside, same era.
Aggregated residual transformations for deep neural networks
S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He · 2017
Cited alongside, same era.
Wild patterns: Ten years after the rise of adversarial machine learning
B. Biggio and F. Roli · 2018
Cited alongside, same era.
Functional map of the world
G. Christie, N. Fendley, J. Wilson, and R. Mukherjee · 2018
Cited alongside, same era.
Senteval: An evaluation toolkit for universal sentence representations
A. Conneau and D. Kiela · 2018
Cited alongside, same era.
Clinically applicable deep learning for diagnosis and referral in retinal disease
An image is worth 16x16 words: Transformers for image recognition at scale
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al · 2020
Later among the works it cites.
Big transfer (bit): General visual representation learning
A. Kolesnikov, L. Beyer, X. Zhai, J. Puigcerver, J. Yung, S. Gelly, and N. Houlsby · 2020
Later among the works it cites.
Do adversarially robust imagenet models transfer better?
H. Salman, A. Ilyas, L. Engstrom, A. Kapoor, and A. Madry · 2020
Later among the works it cites.
Breeds: Benchmarks for subpopulation shift
S. Santurkar, D. Tsipras, and A. Madry · 2020
Later among the works it cites.
Evaluating machine accuracy on imagenet
V. Shankar*, R. Roelofs*, H. Mania, A. Fang, B. Recht, and L. Schmidt · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
J. De Fauw, J. R. Ledsam, B. Romera-Paredes, S. Nikolov, N. Tomasev, S. Blackwell, H. Askham, X. Glorot, B. O’Donoghue, D. Visentin, et al · 2018
Cited alongside, same era.
Comparison of deep transfer learning strategies for digital pathology
R. Mormont, P. Geurts, and R. Marée · 2018
Cited alongside, same era.
A. Rosenfeld, R. Zemel, and J. K. Tsotsos · 2018
Cited alongside, same era.
The inaturalist species classification and detection dataset
G. Van Horn, O. Mac Aodha, Y. Song, Y. Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie · 2018
Cited alongside, same era.
Spottune: transfer learning through adaptive fine-tuning
Y. Guo, H. Shi, A. Kumar, K. Grauman, T. Rosing, and R. Feris · 2019
Cited alongside, same era.
Searching for mobilenetv3
A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan, et al · 2019
Cited alongside, same era.
Do better imagenet models transfer better?
S. Kornblith, J. Shlens, and Q. V. Le · 2019
Cited alongside, same era.
Measuring robustness to natural distribution shifts in image classification
R. Taori, A. Dave, V. Shankar, N. Carlini, B. Recht, and L. Schmidt · 2020
Later among the works it cites.
Co-tuning for transfer learning
K. You, Z. Kou, M. Long, and J. Wang · 2020
Later among the works it cites.
The evolution of out-of-distribution robustness throughout fine-tuning
A. Andreassen, Y. Bahri, B. Neyshabur, and R. Roelofs · 2021
Later among the works it cites.
Wilds: A benchmark of in-the-wild distribution shifts
P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Yasunaga, R. L. Phillips, I. Gao, et al · 2021
Later among the works it cites.
Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization
J. P. Miller, R. Taori, A. Raghunathan, S. Sagawa, P. W. Koh, V. Shankar, P. Liang, Y. Carmon, and L. Schmidt · 2021
Later among the works it cites.
Learning transferable visual models from natural language supervision
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al · 2021
Later among the works it cites.
High-resolution image synthesis with latent diffusion models, 2021
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer · 2021
Later among the works it cites.
Laion-400m: Open dataset of clip-filtered 400 million image-text pairs
C. Schuhmann, R. Vencu, R. Beaumont, R. Kaczmarczyk, C. Mullis, A. Katta, T. Coombes, J. Jitsev, and A. Komatsuzaki · 2021
Later among the works it cites.
Data determines distributional robustness in contrastive language image pre-training (clip)
A. Fang, G. Ilharco, M. Wortsman, Y. Wan, V. Shankar, A. Dave, and L. Schmidt · 2022
Later among the works it cites.
Pre-training without natural images
H. Kataoka, K. Okayasu, A. Matsumoto, E. Yamagata, R. Yamada, N. Inoue, A. Nakamura, and Y. Satoh · 2022
Later among the works it cites.
Quality not quantity: On the interaction between dataset design and robustness of clip
T. Nguyen, G. Ilharco, M. Wortsman, S. Oh, and L. Schmidt · 2022
Later among the works it cites.