2019

Adversarial AutoAugment

Zhang, Xinyu, Wang, Qiang, Zhang, Jian et al.

Understand

Data augmentation (DA) has been widely utilized to improve generalization in training deep neural networks.

Recently, human-designed data augmentation has been gradually replaced by automatically learned augmentation policy.
Through finding the best policy in well-designed search space of data augmentation, AutoAugment can significantly improve validation accuracy on image classification tasks.
However, this approach is not computationally practical for large-scale problems.

Built on

Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J. Williams · 1992
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky and Geoffrey E. Hinton · 2009
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton · 2012
Earlier work this paper cites.
Regularization of neural networks using dropconnect
Li Wan, Matthew Zeiler, Sixin Zhang, Yann LeCun, and Rob Fergus · 2013
Earlier work this paper cites.
Generative adversarial networks
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio · 2014
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba · 2015
Earlier work this paper cites.
Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott E. Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Wide residual networks
Sergey Zagoruyko and Nikos Komodakis · 2016
Earlier work this paper cites.
Neural architecture search with reinforcement learning
Barret Zoph and Quoc V. Le · 2016
Earlier work this paper cites.

Similar

Data augmentation generative adversarial networks
Antreas Antoniou, Amos J. Storkey, and Harrison Edwards · 2017
Cited alongside, same era.
Improved regularization of convolutional neural networks with cutout
Original
Terrance Devries and Graham W. Taylor · 2017
Cited alongside, same era.
Shake-shake regularization
Original
Xavier Gastaldi · 2017
Cited alongside, same era.
Deligan : Generative adversarial networks for diverse and limited data
Swaminathan Gurumurthy, Ravi Kiran Sarvadevabhatla, and Venkatesh Babu Radhakrishnan · 2017
Cited alongside, same era.
Deep pyramidal residual networks
Dongyoon Han, Jiwhan Kim, and Junmo Kim · 2017
Cited alongside, same era.
Population based training of neural networks
Original
Max Jaderberg, Valentin Dalibard, Simon Osindero, Wojciech M. Czarnecki, Jeff Donahue, Ali Razavi, Oriol Vinyals, Tim Green, Iain Dunning, Karen Simonyan, Chrisantha Fernando, and Koray Kavukcuoglu · 2017
Cited alongside, same era.
Smart augmentation - learning an optimal data augmentation strategy
Original
Joseph Lemley, Shabab Bazrafkan, and Peter Corcoran · 2017
Cited alongside, same era.
The effectiveness of data augmentation in image classification using deep learning
Original
Luis Perez and Jason Wang · 2017
Cited alongside, same era.
A bayesian data augmentation approach for learning deep models
Toan Tran, Trung Pham, Gustavo Carneiro, Lyle J. Palmer, and Ian D. Reid · 2017
Cited alongside, same era.
A-fast-rcnn: Hard positive generation via adversary for object detection
Xiaolong Wang, Abhinav Shrivastava, and Abhinav Gupta · 2017
Cited alongside, same era.
Learning transferable architectures for scalable image recognition
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V. Le · 2017
Cited alongside, same era.
Practical network blocks design with Q-learning
Zhao Zhong, Junjie Yan, and Cheng-Lin Liu
Cited in the paper.

Then

Autoaugment: Learning augmentation policies from data
Ekin D. Cubuk, Barret Zoph, Dandelion Mané, Vijay Vasudevan, and Quoc V. Le · 2018
Later among the works it cites.
Synthetic data augmentation using GAN for improved liver lesion classification
Maayan Frid-Adar, Eyal Klang, Michal Amitai, Jacob Goldberger, and Hayit Greenspan · 2018
Later among the works it cites.
IRLAS: inverse reinforcement learning for architecture search
Original
Minghao Guo, Zhao Zhong, Wei Wu, Dahua Lin, and Junjie Yan · 2018
Later among the works it cites.
Bag of tricks for image classification with convolutional neural networks
Original
Tong He, Zhi Zhang, Hang Zhang, Zhongyue Zhang, Junyuan Xie, and Mu Li · 2018
Later among the works it cites.
Data augmentation by pairing samples for images classification
Original
Hiroshi Inoue · 2018
Later among the works it cites.
Jointly optimize data augmentation and network training: Adversarial data augmentation in human pose estimation
Xi Peng, Zhiqiang Tang, Fei Yang, Rogério Schmidt Feris, and Dimitris N. Metaxas · 2018
Later among the works it cites.
Invariant representation learning for robust deep networks
Julian Salazar, Davis Liang, Zhiheng Huang, and Zachary C. Lipton · 2018
Later among the works it cites.
Shakedrop regularization
Original
Yoshihiro Yamada, Masakazu Iwamura, and Koichi Kise · 2018
Later among the works it cites.
Population based augmentation: Efficient learning of augmentation policy schedules
Daniel Ho, Eric Liang, Ion Stoica, Pieter Abbeel, and Xi Chen · 2019
Closest in time.
Augment your batch: better training with larger batches
Original
Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, and Daniel Soudry · 2019
Closest in time.
Online hyper-parameter learning for auto-augmentation strategy
Original
Chen Lin, Minghao Guo, Chuming Li, Wei Wu, Dahua Lin, Wanli Ouyang, and Junjie Yan · 2019
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…