Fetching the paper…
Reading the bibliography…
We identify label errors in the test sets of 10 of the most commonly-used computer vision, natural language, and audio datasets, and subsequently study the potential for these label errors to affect benchmark results.
Maximum likelihood estimation of observer error-rates using the em algorithm
A. P. Dawid and A. M. Skene · 1979
Earlier work this paper cites.
Learning from noisy examples
D. Angluin and P. Laird · 1988
Earlier work this paper cites.
Nist special database 19 handprinted forms and characters database
P. J. Grother · 1995
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
Twenty newsgroups dataset
T. Mitchell · 1999
Earlier work this paper cites.
Caltech-256 object category dataset
G. Griffin, A. Holub, and P. Perona · 2007
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky and G. Hinton · 2009
Earlier work this paper cites.
Learning word vectors for sentiment analysis
A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts · 2011
Earlier work this paper cites.
Classification in the presence of label noise: A survey
B. Frénay and M. Verleysen · 2013
Earlier work this paper cites.
Learning with noisy labels
N. Natarajan, I. S. Dhillon, P. K. Ravikumar, and A. Tewari · 2013
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2014
Earlier work this paper cites.
Image-based recommendations on styles and substitutes
J. McAuley, C. Targett, Q. Shi, and A. van den Hengel · 2015
Earlier work this paper cites.
Learning with symmetric label noise: The importance of being unhinged
B. Van Rooyen, A. Menon, and R. C. Williamson · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Learning deep networks from noisy labels with dropout regularization
I. Jindal, M. Nokleby, and X. Chen · 2016
Earlier work this paper cites.
Loss factorization, weakly supervised learning and label noise robustness
G. Patrini, F. Nielsen, R. Nock, and M. Carioni · 2016
Earlier work this paper cites.
Data programming: Creating large training sets, quickly
A. J. Ratner, C. M. De Sa, S. Wu, D. Selsam, and C. Ré · 2016
Cited alongside, same era.
A closer look at memorization in deep networks
D. Arpit, S. Jastrzębski, N. Ballas, D. Krueger, E. Bengio, M. S. Kanwal, T. Maharaj, A. Fischer, A. Courville, Y. Bengio, et al · 2017
Cited alongside, same era.
Audio set: An ontology and human-labeled dataset for audio events
J. F. Gemmeke, D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter · 2017
Cited alongside, same era.
Training deep neural-networks using a noise adaptation layer
J. Goldberger and E. Ben-Reuven · 2017
Cited alongside, same era.
A neural representation of sketch drawings
D. Ha and D. Eck · 2017
Cited alongside, same era.
Webvision database: Visual learning and understanding from web data
Learning transferable architectures for scalable image recognition
B. Zoph, V. Vasudevan, J. Shlens, and Q. V. Le · 2018
Later among the works it cites.
Understanding and utilizing deep neural networks trained with noisy labels
P. Chen, B. B. Liao, G. Chen, and S. Zhang · 2019
Later among the works it cites.
Selective brain damage: Measuring the disparate impact of model pruning
S. Hooker, A. Courville, Y. Dauphin, and A. Frome · 2019
Later among the works it cites.
Understanding generalization through visualizations
W. R. Huang, Z. Emam, M. Goldblum, L. Fowl, J. K. Terry, F. Huang, and T. Goldstein · 2019
Later among the works it cites.
Do imagenet classifiers generalize to imagenet?
B. Recht, R. Roelofs, L. Schmidt, and V. Shankar · 2019
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
W. Li, L. Wang, W. Li, E. Agustsson, and L. Van Gool · 2017
Cited alongside, same era.
Learning with confident examples: Rank pruning for robust classification with noisy labels
C. G. Northcutt, T. Wu, and I. L. Chuang · 2017
Cited alongside, same era.
Making deep neural networks robust to label noise: A loss correction approach
G. Patrini, A. Rozza, A. Krishna Menon, R. Nock, and L. Qu · 2017
Cited alongside, same era.
Deep learning is robust to massive label noise
D. Rolnick, A. Veit, S. Belongie, and N. Shavit · 2017
Cited alongside, same era.
Revisiting unreasonable effectiveness of data in deep learning era
C. Sun, A. Shrivastava, S. Singh, and A. Gupta · 2017
Cited alongside, same era.
Improving crowdsourced label quality using noise correction
J. Zhang, V. S. Sheng, T. Li, and X. Wu · 2017
Cited alongside, same era.
Using trusted data to train deep networks on labels corrupted by severe noise
D. Hendrycks, M. Mazeika, D. Wilson, and K. Gimpel · 2018
Cited alongside, same era.
L_dmi: A novel information-theoretic loss function for training deep nets robust to label noise
Y. Xu, P. Cao, Y. Kong, and Y. Wang · 2019
Later among the works it cites.
A survey on deep learning with noisy labels: How to train your model when you cannot trust on the annotations?
F. R. Cordeiro and G. Carneiro · 2020
Later among the works it cites.
Improving generalization by controlling label-noise information in neural network weights
H. Harutyunyan, K. Reing, G. Ver Steeg, and A. Galstyan · 2020
Later among the works it cites.
Beyond synthetic noise: Deep learning on controlled noisy labels
L. Jiang, D. Huang, M. Liu, and W. Yang · 2020
Later among the works it cites.
Evaluating machine accuracy on ImageNet
V. Shankar, R. Roelofs, H. Mania, A. Fang, B. Recht, and L. Schmidt · 2020
Later among the works it cites.
From imagenet to image classification: Contextualizing progress on benchmarks
D. Tsipras, S. Santurkar, L. Engstrom, A. Ilyas, and A. Madry · 2020
Later among the works it cites.
CleanML: a study for evaluating the impact of data cleaning on ml classification tasks
P. Li, X. Rao, J. Blase, Y. Zhang, X. Chu, and C. Zhang · 2021
Closest in time.
Confident learning: Estimating uncertainty in dataset labels
C. G. Northcutt, L. Jiang, and I. Chuang · 2021
Closest in time.
"Everyone wants to do the model work, not the data work": Data cascades in high-stakes ai
N. Sambasivan, S. Kapania, H. Highfill, D. Akrong, P. Paritosh, and L. M. Aroyo · 2021
Closest in time.
Fair classification with group-dependent label noise
J. Wang, Y. Liu, and C. Levy · 2021
Closest in time.
Training convolutional networks with noisy labels
S. Sukhbaatar, J. Bruna, M. Paluri, L. Bourdev, and R. Fergus · 2080
Closest in time.