2020

GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning

Killamsetty, Krishnateja, Sivasubramanian, Durga, Ramakrishnan, Ganesh et al.

Understand

Large scale machine learning and deep models are extremely data-hungry.

Unfortunately, obtaining large amounts of labeled data is expensive, and training state-of-the-art models (with hyperparameter tuning) requires significant computing resources and time.
Secondly, real-world data is noisy and imbalanced.
As a result, several recent papers try to make the training process more efficient and robust.

Built on

Submodularity in data subset selection and active learning
Wei, K.; Iyer, R.; and Bilmes, J. 2015 · 1963
Earlier work this paper cites.
Accelerated greedy algorithms for maximizing submodular set functions
Minoux, M. 1978 · 1978
Earlier work this paper cites.
An analysis of approximations for maximizing submodular set functions—I
Nemhauser, G. L.; Wolsey, L. A.; and Fisher, M. L. 1978 · 1978
Earlier work this paper cites.
Backpropagation applied to handwritten zip code recognition
LeCun, Y.; Boser, B.; Denker, J. S.; Henderson, D.; Howard, R. E.; Hubbard, W.; and Jackel, L. D. 1989 · 1989
Earlier work this paper cites.
LIBSVM: a library for support vector machines,” 2001. Software available at http://www. csie. ntu. edu. tw/˜ cjlin/libsvm
Chang, C.-C.; and Lin, C.-J. 2001 · 2001
Earlier work this paper cites.
On coresets for k-means and k-median clustering
Har-Peled, S.; and Mazumdar, S. 2004 · 2004
Earlier work this paper cites.
Submodular functions and optimization
Fujishige, S. 2005 · 2005
Earlier work this paper cites.
Submodular combinatorial information measures with applications in machine learning
Original
Iyer, R.; Khargoankar, N.; Bilmes, J.; and Asanani, H. 2020 · 2006
Earlier work this paper cites.
Non-monotone submodular maximization under matroid and knapsack constraints
Lee, J.; Mirrokni, V. S.; Nagarajan, V.; and Sviridenko, M. 2009 · 2009
Earlier work this paper cites.
Active learning literature survey
Settles, B. 2009 · 2009
Earlier work this paper cites.
Coresets, sparse greedy approximation, and the Frank-Wolfe algorithm
Clarkson, K. L. 2010 · 2010
Earlier work this paper cites.
Submodular meets spectral: Greedy algorithms for subset selection, sparse approximation and dictionary selection
Original
Das, A.; and Kempe, D. 2011 · 2011
Earlier work this paper cites.
Scikit-learn: Machine Learning in Python
Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; Vanderplas, J.; Passos, A.; Cournapeau, D.; Brucher, M.; Perrot, M.; and Duchesnay, E. 2011 · 2011
Earlier work this paper cites.

Similar

Submodular maximization with cardinality constraints
Buchbinder, N.; Feldman, M.; Naor, J.; and Schwartz, R. 2014 · 2014
Cited alongside, same era.
Submodularity for data selection in machine translation
Kirchhoff, K.; and Bilmes, J. 2014 · 2014
Cited alongside, same era.
Lazier than lazy greedy
Original
Mirzasoleiman, B.; Badanidiyuru, A.; Karbasi, A.; Vondrák, J.; and Krause, A. 2014 · 2014
Cited alongside, same era.
A submodular optimization approach to sentence set selection
Shinohara, Y. 2014 · 2014
Cited alongside, same era.
Fast multi-stage submodular maximization
Wei, K.; Iyer, R.; and Bilmes, J. 2014 · 2014
Cited alongside, same era.
Submodular subset selection for large-scale speech training data
Wei, K.; Liu, Y.; Kirchhoff, K.; Bartels, C.; and Bilmes, J. 2014a · 2014
Cited alongside, same era.
Unsupervised submodular subset selection for speech data
Wei, K.; Liu, Y.; Kirchhoff, K.; and Bilmes, J. 2014b · 2014
Cited alongside, same era.
Summarization of multi-document topic hierarchies using submodular mixtures
Bairi, R.; Iyer, R.; Ramakrishnan, G.; and Bilmes, J. 2015 · 2015
Cited alongside, same era.
Submodular optimization and machine learning: Theoretical results, unifying and scalable algorithms, and applications
Iyer, R. K. 2015 · 2015
Cited alongside, same era.
SVitchboard II and FiSVer I: High-quality limited-complexity corpora of conversational English speech
Liu, Y.; Iyer, R.; Kirchhoff, K.; and Bilmes, J. 2015 · 2015
Cited alongside, same era.
Deep residual learning for image recognition
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016 · 2016
Cited alongside, same era.
UCI Machine Learning Repository
Dua, D.; and Graff, C. 2017 · 2017
Cited alongside, same era.

Then

Automatic differentiation in PyTorch
Paszke, A.; Gross, S.; Chintala, S.; Chanan, G.; Yang, E.; DeVito, Z.; Lin, Z.; Desmaison, A.; Antiga, L.; and Lerer, A. 2017 · 2017
Later among the works it cites.
Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent
Campbell, T.; and Broderick, T. 2018 · 2018
Later among the works it cites.
On the Network Visibility Problem
Original
Gatmiry, K.; and Gomez-Rodriguez, M. 2018 · 2018
Later among the works it cites.
Co-teaching: Robust training of deep neural networks with extremely noisy labels
Han, B.; Yao, Q.; Yu, X.; Niu, G.; Xu, M.; Hu, W.; Tsang, I.; and Sugiyama, M. 2018 · 2018
Later among the works it cites.
Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels
Jiang, L.; Zhou, Z.; Leung, T.; Li, L.-J.; and Fei-Fei, L. 2018 · 2018
Later among the works it cites.
Learning to Reweight Examples for Robust Deep Learning
Ren, M.; Zeng, W.; Yang, B.; and Urtasun, R. 2018 · 2018
Later among the works it cites.
Active Learning for Convolutional Neural Networks: A Core-Set Approach
Sener, O.; and Savarese, S. 2018 · 2018
Later among the works it cites.
Generalized cross entropy loss for training deep neural networks with noisy labels
Zhang, Z.; and Sabuncu, M. 2018 · 2018
Later among the works it cites.
Learning from less data: A unified data subset selection and active learning framework for computer vision
Kaushal, V.; Iyer, R.; Kothawade, S.; Mahadev, R.; Doctor, K.; and Ramakrishnan, G. 2019 · 2019
Later among the works it cites.
Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds
Ash, J. T.; Zhang, C.; Krishnamurthy, A.; Langford, J.; and Agarwal, A. 2020 · 2020
Closest in time.
Core-Sets: Updated Survey
Feldman, D. 2020 · 2020
Closest in time.
Coresets for Data-efficient Training of Machine Learning Models
Mirzasoleiman, B.; Bilmes, J.; and Leskovec, J. 2020 · 2020
Closest in time.
Distributed submodular maximization: Identifying representative elements in massive data
Mirzasoleiman, B.; Karbasi, A.; Sarkar, R.; and Krause, A. 2013 · 2057
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…