Fetching the paper…
Reading the bibliography…
Pre-training is a widely used approach to develop models that are robust to distribution shifts.
“Improving predictive inference under covariate shift by weighting the log-likelihood function”
Hidetoshi Shimodaira · 2000
Earlier work this paper cites.
“The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization”, 2020
Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt and Justin Gilmer · 2006
Earlier work this paper cites.
“Imagenet: A large-scale hierarchical image database”
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li and Li Fei-Fei · 2009
Earlier work this paper cites.
“Theory of convex optimization for machine learning”
Sébastien Bubeck · 2014
Earlier work this paper cites.
“CNN features off-the-shelf: an astounding baseline for recognition”
Ali Sharif, Hossein Azizpour, Josephine Sullivan and Stefan Carlsson · 2014
Earlier work this paper cites.
“Deep Residual Learning for Image Recognition”
Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun · 2015
Earlier work this paper cites.
“Deep Learning Face Attributes in the Wild”
Ziwei Liu, Ping Luo, Xiaogang Wang and Xiaoou Tang · 2015
Earlier work this paper cites.
“Transfer learning from deep features for remote sensing and poverty mapping”
Michael Xie, Neal Jean, Marshall Burke, David Lobell and Stefano Ermon · 2016
Earlier work this paper cites.
“On calibration of modern neural networks”
Chuan Guo, Geoff Pleiss, Yu Sun and Kilian Weinberger · 2017
Earlier work this paper cites.
“Revisiting unreasonable effectiveness of data in deep learning era”
Chen Sun, Abhinav Shrivastava, Saurabh Singh and Abhinav Gupta · 2017
Earlier work this paper cites.
“From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge”
Peter Bandi, Oscar Geessink, Quirine Manson, Marcory Van, Maschenka Balkenhol, Meyke Hermsen, Babak Bejnordi, Byungjae Lee, Kyunghyun Paeng and Aoxiao Zhong · 2018
Earlier work this paper cites.
“Functional Map of the World”
Gordon Christie, Neil Fendley, James Wilson and Ryan Mukherjee · 2018
Earlier work this paper cites.
“Progressive Growing of GANs for Improved Quality, Stability, and Variation”
Tero Karras, Timo Aila, Samuli Laine and Jaakko Lehtinen · 2018
Earlier work this paper cites.
“Detecting and correcting for label shift with black box predictors”
Zachary Lipton, Yu-Xiang Wang and Alexander Smola · 2018
Earlier work this paper cites.
“Exploring the limits of weakly supervised pretraining”
Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe and Laurens van Maaten · 2018
Earlier work this paper cites.
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani and David Lopez-Paz · 2019
Earlier work this paper cites.
“What is the effect of importance weighting in deep learning?”
Jonathon Byrd and Zachary Lipton · 2019
Earlier work this paper cites.
“ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models”
Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum and Boris Katz · 2019
Earlier work this paper cites.
“Rethinking imagenet pre-training”
Kaiming He, Ross Girshick and Piotr Dollár · 2019
Earlier work this paper cites.
“Using pre-training can improve model robustness and uncertainty”
Dan Hendrycks, Kimin Lee and Mantas Mazeika · 2019
Earlier work this paper cites.
“Big Transfer (BiT): General Visual Representation Learning”
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly and Neil Houlsby · 2019
Earlier work this paper cites.
“Do better imagenet models transfer better?”
Simon Kornblith, Jonathon Shlens and Quoc Le · 2019
Earlier work this paper cites.
“Do ImageNet Classifiers Generalize to ImageNet?”
Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt and Vaishaal Shankar · 2019
Cited alongside, same era.
“Do Image Classifiers Generalize Across Time?”
Vaishaal Shankar, Achal Dave, Rebecca Roelofs, Deva Ramanan, Benjamin Recht and Ludwig Schmidt · 2019
Cited alongside, same era.
“Learning robust global representations by penalizing local predictive power”
Haohan Wang, Songwei Ge, Eric Xing and Zachary Lipton · 2019
Cited alongside, same era.
“PyTorch Image Models”
Ross Wightman · 2019
Cited alongside, same era.
“Model patching: Closing the subgroup performance gap with data augmentation”
Karan Goel, Albert Gu, Yixuan Li and Christopher Ré · 2020
Cited alongside, same era.
“Scaling up visual and vision-language representation learning with noisy text supervision”
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li and Tom Duerig · 2021
Later among the works it cites.
“CheXtransfer: performance and parameter efficiency of ImageNet models for chest X-Ray interpretation”
Alexander Ke, William Ellsworth, Oishi Banerjee, Andrew Ng and Pranav Rajpurkar · 2021
Later among the works it cites.
“Just Train Twice: Improving Group Robustness without Training Group Information”
Evan Liu, Behzad Haghgoo, Annie. Chen, Aditi Raghunathan, Pang Koh, Shiori Sagawa, Percy Liang and Chelsea Finn · 2021
Later among the works it cites.
“Accuracy on the line: on the strong correlation between out-of-distribution and in-distribution generalization”
John Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Koh, Vaishaal Shankar, Percy Liang, Yair Carmon and Ludwig Schmidt · 2021
Later among the works it cites.
“Learning transferable visual models from natural language supervision”
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge and Felix Wichmann · 2020
Cited alongside, same era.
“In search of lost domain generalization”
Ishaan Gulrajani and David Lopez-Paz · 2020
Cited alongside, same era.
“Pretrained transformers improve out-of-distribution robustness”
Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan and Dawn Song · 2020
Cited alongside, same era.
“WILDS: A Benchmark of in-the-Wild Distribution Shifts”
Pang Koh, Shiori Sagawa, Henrik Marklund, Sang Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Phillips and Sara Beery · 2020
Cited alongside, same era.
“Maskgan: Towards diverse and interactive facial image manipulation”
Cheng-Han Lee, Ziwei Liu, Lingyun Wu and Ping Luo · 2020
Cited alongside, same era.
“The effect of natural distribution shift on question answering models”
John Miller, Karl Krauth, Benjamin Recht and Ludwig Schmidt · 2020
Cited alongside, same era.
“Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization”
Shiori Sagawa, Pang Koh, Tatsunori. Hashimoto and Percy Liang · 2020
Cited alongside, same era.
Alec Radford, Jong Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin and Jack Clark · 2021
Later among the works it cites.
“How to train your vit? data, augmentation, and regularization in vision transformers”
Andreas Steiner, Alexander Kolesnikov, Xiaohua Zhai, Ross Wightman, Jakob Uszkoreit and Lucas Beyer · 2021
Later among the works it cites.
“Breeds: Benchmarks for subpopulation shift”
Shibani Santurkar, Dimitris Tsipras and Aleksander Madry · 2021
Later among the works it cites.
“A fine-grained analysis on distribution shift”
Olivia Wiles, Sven Gowal, Florian Stimberg, Sylvestre Alvise-Rebuffi, Ira Ktena, Krishnamurthy Dvijotham and Taylan Cemgil · 2021
Later among the works it cites.
“Robust fine-tuning of zero-shot models”
Mitchell Wortsman, Gabriel Ilharco, Jong Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo-Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong and Ludwig Schmidt · 2021
Later among the works it cites.
“Simple data balancing achieves competitive worst-group-accuracy”
Badr Idrissi, Martin Arjovsky, Mohammad Pezeshki and David Lopez-Paz · 2022
Later among the works it cites.
“Last layer re-training is sufficient for robustness to spurious correlations”
Polina Kirichenko, Pavel Izmailov and Andrew Wilson · 2022
Later among the works it cites.
“Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution”
Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma and Percy Liang · 2022
Later among the works it cites.
“ffcv”, https://github.com/libffcv/ffcv/ , 2022
Guillaume Leclerc, Andrew Ilyas, Logan Engstrom, Sung Park, Hadi Salman and Aleksander Madry · 2022
Later among the works it cites.
“A convnet for the 2020s”
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell and Saining Xie · 2022
Later among the works it cites.
Ziquan Liu, Yi Xu, Yuanhong Xu, Qi Qian, Hao Li, Rong Jin, Xiangyang Ji and Antoni Chan · 2022
Later among the works it cites.
“High-resolution image synthesis with latent diffusion models”
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser and Björn Ommer · 2022
Later among the works it cites.
“Laion-5b: An open large-scale dataset for training next generation image-text models”
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis and Mitchell Wortsman · 2022
Later among the works it cites.
“When does Bias Transfer in Transfer Learning?”
Hadi Salman, Saachi Jain, Andrew Ilyas, Logan Engstrom, Eric Wong and Aleksander Madry · 2022
Later among the works it cites.
“Ood-bench: Quantifying and understanding two dimensions of out-of-distribution generalization”
Nanyang Ye, Kaican Li, Haoyue Bai, Runpeng Yu, Lanqing Hong, Fengwei Zhou, Zhenguo Li and Jun Zhu · 2022
Later among the works it cites.
“Instructpix2pix: Learning to follow image editing instructions”
Tim Brooks, Aleksander Holynski and Alexei Efros · 2023
Later among the works it cites.
“On the Connection between Pre-training Data Diversity and Fine-tuning Robustness”
Vivek Ramanujan, Thao Nguyen, Sewoong Oh, Ludwig Schmidt and Ali Farhadi · 2023
Later among the works it cites.
“Change is Hard: A Closer Look at Subpopulation Shift”
Yuzhe Yang, Haoran Zhang, Dina Katabi and Marzyeh Ghassemi · 2023
Later among the works it cites.