2019

Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels

Jiang, Lu, Huang, Di, Liu, Mason et al.

Understand

Performing controlled experiments on noisy data is essential in understanding deep learning across noise levels.

Due to the lack of suitable datasets, previous research has only examined deep learning on controlled synthetic label noise, and real-world label noise has never been studied in a controlled setting.
This paper makes three contributions.
First, we establish the first benchmark of controlled real-world label noise from the web.

Built on

Imagenet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Krizhevsky, A. and Hinton, G · 2009
Earlier work this paper cites.
Self-paced learning for latent variable models
Kumar, M. P., Packer, B., and Koller, D · 2010
Earlier work this paper cites.
Label-noise robust logistic regression and its applications
Bootkrajang, J. and Kabán, A · 2012
Earlier work this paper cites.
Learning to label aerial images from noisy data
Mnih, V. and Hinton, G. E · 2012
Earlier work this paper cites.
Collecting a large-scale dataset of fine-grained cars
Krause, J., Deng, J., Stark, M., and Fei-Fei, L · 2013
Earlier work this paper cites.
Self-paced learning with diversity
Jiang, L., Meng, D., Yu, S.-I., Lan, Z., Shan, S., and Hauptmann, A · 2014
Earlier work this paper cites.
Training deep neural networks on noisy labels with bootstrapping
Original
Reed, S., Lee, H., Anguelov, D., Szegedy, C., Erhan, D., and Rabinovich, A · 2014
Earlier work this paper cites.
Dropout: a simple way to prevent neural networks from overfitting
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R · 2014
Earlier work this paper cites.
Webly supervised learning of convolutional networks
Chen, X. and Gupta, A · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Ioffe, S. and Szegedy, C · 2015
Earlier work this paper cites.
Self-paced curriculum learning
Jiang, L., Meng, D., Zhao, Q., Shan, S., and Hauptmann, A. G · 2015
Earlier work this paper cites.
Learning with symmetric label noise: The importance of being unhinged
Van Rooyen, B., Menon, A., and Williamson, R. C · 2015
Earlier work this paper cites.
Learning from massive noisy labeled data for image classification
Xiao, T., Xia, T., Yang, Y., Huang, C., and Wang, X · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Ioffe, S. and Szegedy, C · 2015
Earlier work this paper cites.
Learning from massive noisy labeled data for image classification
Xiao, T., Xia, T., Yang, Y., Huang, C., and Wang, X · 2015
Earlier work this paper cites.
Auxiliary image regularization for deep cnns with noisy labels
Azadi, S., Feng, J., Jegelka, S., and Darrell, T · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
The unreasonable effectiveness of noisy data for fine-grained recognition
Krause, J., Sapp, B., Howard, A., Zhou, H., Toshev, A., Duerig, T., Philbin, J., and Fei-Fei, L · 2016
Earlier work this paper cites.
Learning to detect concepts from webly-labeled video data
Liang, J., Jiang, L., Meng, D., and Hauptmann, A. G · 2016
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z · 2016
Earlier work this paper cites.
Matching networks for one shot learning
Vinyals, O., Blundell, C., Lillicrap, T., Wierstra, D., et al · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S., and Sun, J · 2016
Earlier work this paper cites.
Rethinking the inception architecture for computer vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z · 2016
Earlier work this paper cites.
A closer look at memorization in deep networks
Arpit, D., Jastrzkebski, S., Ballas, N., Krueger, D., Bengio, E., Kanwal, M. S., Maharaj, T., Fischer, A., Courville, A., Bengio, Y., et al · 2017
Earlier work this paper cites.

Similar

Self-paced learning: an implicit regularization perspective
Fan, Y., He, R., Liang, J., and Hu, B · 2017
Cited alongside, same era.
Training deep neural-networks using a noise adaptation layer
Goldberger, J. and Ben-Reuven, E · 2017
Cited alongside, same era.
Regularizing deep neural networks by noise: its interpretation and optimization
Noh, H., You, T., Mun, J., and Han, B · 2017
Cited alongside, same era.
Making deep neural networks robust to label noise: A loss correction approach
Patrini, G., Rozza, A., Krishna Menon, A., Nock, R., and Qu, L · 2017
Cited alongside, same era.
Deep learning is robust to massive label noise
Original
Rolnick, D., Veit, A., Belongie, S., and Shavit, N · 2017
Cited alongside, same era.
Inception-v4, inception-resnet and the impact of residual connections on learning
Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. A · 2017
Cited alongside, same era.
Toward robustness against label noise in training deep discriminative neural networks
Vahdat, A · 2017
Cited alongside, same era.
Learning from noisy large-scale datasets with minimal supervision
Veit, A., Alldrin, N., Chechik, G., Krasin, I., Gupta, A., and Belongie, S · 2017
Cited alongside, same era.
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., and Vinyals, O · 2017
Cited alongside, same era.
Accurate, large minibatch sgd: Training imagenet in 1 hour
Original
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., and He, K · 2017
Cited alongside, same era.
Webvision database: Visual learning and understanding from web data
Original
Li, W., Wang, L., Li, W., Agustsson, E., and Van Gool, L · 2017
Cited alongside, same era.
Inception-v4, inception-resnet and the impact of residual connections on learning
Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. A · 2017
Cited alongside, same era.

Then

Mobilenetv2: Inverted residuals and linear bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C · 2018
Later among the works it cites.
Unsupervised label noise modeling and loss correction
Arazo, E., Ortego, D., Albert, P., O’Connor, N. E., and McGuinness, K · 2019
Closest in time.
On symmetric losses for learning from corrupted labels
Charoenphakdee, N., Lee, J., and Sugiyama, M · 2019
Closest in time.
Understanding and utilizing deep neural networks trained with noisy labels
Chen, P., Liao, B., Chen, G., and Zhang, S · 2019
Closest in time.
Robust neural machine translation with doubly adversarial inputs
Cheng, Y., Jiang, L., and Macherey, W · 2019
Closest in time.
Benchmarking neural network robustness to common corruptions and perturbations
Hendrycks, D. and Dietterich, T · 2019
Closest in time.
O2u-net: A simple noisy label detection approach for deep neural networks
Huang, J., Qu, L., Jia, R., and Zhao, B · 2019
Closest in time.
Do better imagenet models transfer better?
Kornblith, S., Shlens, J., and Le, Q. V · 2019
Closest in time.
Robust inference via generative classifiers for handling noisy labels
Lee, K., Yun, S., Lee, K., Lee, H., Li, B., and Shin, J · 2019
Closest in time.
Learning to learn from noisy labeled data
Li, J., Wong, Y., Zhao, Q., and Kankanhalli, M. S · 2019
Closest in time.
Confident learning: Estimating uncertainty in dataset labels
Original
Northcutt, C. G., Jiang, L., and Chuang, I. L · 2019
Closest in time.
Fast rates for a knn classifier robust to unknown asymmetric label noise
Reeve, H. W. and Kabán, A · 2019
Closest in time.
Data parameters: A new family of parameters for learning a differentiable curriculum
Saxena, S., Tuzel, O., and DeCoste, D · 2019
Closest in time.
Combinatorial inference against label noise
Seo, P. H., Kim, G., and Han, B · 2019
Closest in time.
Meta-weight-net: Learning an explicit mapping for sample weighting
Shu, J., Xie, Q., Yi, L., Zhao, Q., Zhou, S., Xu, Z., and Meng, D · 2019
Closest in time.
Selfie: Refurbishing unclean samples for robust deep learning
Song, H., Kim, M., and Lee, J.-G · 2019
Closest in time.
Efficientnet: Rethinking model scaling for convolutional neural networks
Tan, M. and Le, Q. V · 2019
Closest in time.
Theoretically principled trade-off between robustness and accuracy
Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., and Jordan, M. I · 2019
Closest in time.
Do better imagenet models transfer better?
Kornblith, S., Shlens, J., and Le, Q. V · 2019
Closest in time.
Efficientnet: Rethinking model scaling for convolutional neural networks
Tan, M. and Le, Q. V · 2019
Closest in time.
Advaug: Robust adversarial augmentation for neural machine translation
Cheng, Y., Jiang, L., Macherey, W., and Eisenstein, J · 2020
Closest in time.
Dividemix: Learning with noisy labels as semi-supervised learning
Li, J., Socher, R., and Hoi, S. C · 2020
Closest in time.
Simaug: Learning robust representations from simulation for trajectory prediction
Liang, J., Jiang, L., and Hauptmann, A · 2020
Closest in time.
Distilling effective supervision from severe label noise
Zhang, Z., Zhang, H., Arik, S. O., Lee, H., and Pfister, T · 2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…