Fetching the paper…
Reading the bibliography…
We propose a metric -- Projection Norm -- to predict a model's performance on out-of-distribution (OOD) data without access to ground truth labels.
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., and Friedman, J · 2001
Earlier work this paper cites.
Co-validation: Using model disagreement on unlabeled data to validate classification algorithms
Madani, O., Pennock, D., and Flake, G · 2004
Earlier work this paper cites.
Analysis of representations for domain adaptation
Ben-David, S., Blitzer, J., Crammer, K., Pereira, F., et al · 2007
Earlier work this paper cites.
Measuring robustness to natural distribution shifts in image classification
Taori, R., Dave, A., Shankar, V., Carlini, N., Recht, B., and Schmidt, L · 2007
Earlier work this paper cites.
Dataset shift in machine learning
Quiñonero-Candela, J., Sugiyama, M., Schwaighofer, A., and Lawrence, N. D · 2008
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A., Hinton, G., et al · 2009
Earlier work this paper cites.
A theory of learning from different domains
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., and Vaughan, J. W · 2010
Earlier work this paper cites.
Unsupervised supervised learning i: Estimating classification and regression errors without labels
Donmez, P., Lebanon, G., and Balasubramanian, K · 2010
Earlier work this paper cites.
Domain adaptation via transfer component analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., and Yang, Q · 2010
Earlier work this paper cites.
Unsupervised supervised learning ii: Margin-based classification without labels
Balasubramanian, K., Donmez, P., and Lebanon, G · 2011
Earlier work this paper cites.
Explaining and harnessing adversarial examples
Goodfellow, I. J., Shlens, J., and Szegedy, C · 2014
Earlier work this paper cites.
A sick cure for the evaluation of compositional distributional semantic models
Marelli, M., Menini, S., Baroni, M., Bentivogli, L., Bernardi, R., Zamparelli, R., et al · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Simonyan, K. and Zisserman, A · 2014
Earlier work this paper cites.
A large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., and Manning, C. D · 2015
Earlier work this paper cites.
Learning transferable features with deep adaptation networks
Long, M., Cao, Y., Wang, J., and Jordan, M · 2015
Earlier work this paper cites.
Domain-adversarial training of neural networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., and Lempitsky, V · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
A baseline for detecting misclassified and out-of-distribution examples in neural networks
Hendrycks, D. and Gimpel, K · 2016
Earlier work this paper cites.
Sgdr: Stochastic gradient descent with warm restarts
Loshchilov, I. and Hutter, F · 2016
Earlier work this paper cites.
Unsupervised risk estimation using only conditional independence structure
Steinhardt, J. and Liang, P. S · 2016
Earlier work this paper cites.
Adversarial examples are not easily detected: Bypassing ten detection methods
Carlini, N. and Wagner, D · 2017
Earlier work this paper cites.
Selective classification for deep neural networks
Geifman, Y. and El-Yaniv, R · 2017
Earlier work this paper cites.
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q · 2017
Cited alongside, same era.
Adversarial machine learning at scale
Kurakin, A., Goodfellow, I. J., and Bengio, S · 2017
Cited alongside, same era.
Decoupled weight decay regularization
Loshchilov, I. and Hutter, F · 2017
Cited alongside, same era.
Adversarial discriminative domain adaptation
Tzeng, E., Hoffman, J., Saenko, K., and Darrell, T · 2017
Cited alongside, same era.
A broad-coverage challenge corpus for sentence understanding through inference
Williams, A., Nangia, N., and Bowman, S. R · 2017
Cited alongside, same era.
The marginal value of adaptive gradient methods in machine learning
Gradient descent optimizes over-parameterized deep relu networks
Zou, D., Cao, Y., Zhou, D., and Gu, Q · 2019
Later among the works it cites.
Benign overfitting in linear regression
Bartlett, P. L., Long, P. M., Lugosi, G., and Tsigler, A · 2020
Later among the works it cites.
Estimating generalization under distribution shifts via domain-invariant representations
Chuang, C.-Y., Torralba, A., and Jegelka, S · 2020
Later among the works it cites.
Surprises in high-dimensional ridgeless least squares interpolation, 2020
Hastie, T., Montanari, A., Rosset, S., and Tibshirani, R. J · 2020
Later among the works it cites.
Pretrained transformers improve out-of-distribution robustness
Hendrycks, D., Liu, X., Wallace, E., Dziedzic, A., Krishnan, R., and Song, D · 2020
Later among the works it cites.
The effect of natural distribution shift on question answering models
Miller, J., Krauth, K., Recht, B., and Schmidt, L · 2020
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Wilson, A. C., Roelofs, R., Stern, M., Srebro, N., and Recht, B · 2017
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K · 2018
Cited alongside, same era.
Imagenet-trained cnns are biased towards texture; increasing shape bias improves accuracy and robustness
Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., and Brendel, W · 2018
Cited alongside, same era.
Breaking nli systems with sentences that require simple lexical inferences
Glockner, M., Shwartz, V., and Goldberg, Y · 2018
Cited alongside, same era.
Neural tangent kernel: Convergence and generalization in neural networks
Jacot, A., Gabriel, F., and Hongler, C · 2018
Cited alongside, same era.
A simple unified framework for detecting out-of-distribution samples and adversarial attacks
Lee, K., Lee, K., Lee, H., and Shin, J · 2018
Cited alongside, same era.
Enhancing the reliability of out-of-distribution image detection in neural networks
Liang, S., Li, Y., and Srikant, R · 2018
Cited alongside, same era.
Later among the works it cites.
Distributional generalization: A new kind of generalization
Nakkiran, P. and Bansal, Y · 2020
Later among the works it cites.
Test-time training with self-supervision for generalization under distribution shifts
Sun, Y., Wang, X., Liu, Z., Miller, J., Efros, A., and Hardt, M · 2020
Later among the works it cites.
Benign overfitting in ridge regression
Tsigler, A. and Bartlett, P. L · 2020
Later among the works it cites.
The curse of performance instability in analysis datasets: Consequences, source, and suggestions
Zhou, X., Nie, Y., Tan, H., and Bansal, M · 2020
Later among the works it cites.
Mandoline: Model evaluation under distribution shift
Chen, M., Goel, K., Sohoni, N. S., Poms, F., Fatahalian, K., and Ré, C · 2021
Later among the works it cites.
Are labels always necessary for classifier accuracy evaluation?
Deng, W. and Zheng, L · 2021
Later among the works it cites.
What does rotation prediction tell us about classifier accuracy under varying testing environments?
Deng, W., Gould, S., and Zheng, L · 2021
Later among the works it cites.
Predicting with confidence on unseen distributions
Guillory, D., Shankar, V., Ebrahimi, S., Darrell, T., and Schmidt, L · 2021
Later among the works it cites.
Assessing generalization of sgd via disagreement
Jiang, Y., Nagarajan, V., Baek, C., and Kolter, J. Z · 2021
Later among the works it cites.
Uniform convergence of interpolators: Gaussian width, norm bounds and benign overfitting
Koehler, F., Zhou, L., Sutherland, D. J., and Srebro, N · 2021
Later among the works it cites.
Wilds: A benchmark of in-the-wild distribution shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al · 2021
Later among the works it cites.
Breeds: Benchmarks for subpopulation shift
Santurkar, S., Tsipras, D., and Madry, A · 2021
Later among the works it cites.
Do image classifiers generalize across time?
Shankar, V., Dave, A., Roelofs, R., Ramanan, D., Recht, B., and Schmidt, L · 2021
Later among the works it cites.
Covariate shift in high-dimensional random feature regression
Tripuraneni, N., Adlam, B., and Pennington, J · 2021
Later among the works it cites.
Unsupervised out-of-domain detection via pre-trained transformers
Xu, K., Ren, T., Zhang, S., Feng, Y., and Xiong, C · 2021
Later among the works it cites.
Leveraging unlabeled data to predict out-of-distribution performance
Garg, S., Balakrishnan, S., Lipton, Z. C., Neyshabur, B., and Sedghi, H · 2022
Closest in time.