Built on

Maurice Sion et al., On general minimax theorems. , Pacific Journal of mathematics 8
1958
Earlier work this paper cites.
Yehoram Gordon, On milman’s inequality and random subspaces which escape through a mesh in ℝ n \mathbb{R}^{n} , Geometric aspects of functional analysis, Springer, 1988, pp. 84–106
1988
Earlier work this paper cites.
Friedrich Liese and Klaus-J. Miescke, Statistical decision theory: Estimation, testing, and selection , Springer Science & Business Media, 2008
2008
Earlier work this paper cites.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli, Evasion attacks against machine learning at test time , Joint European conference on machine learning and knowledge discovery in databases, Springer, 2013, pp. 387–402
2013
Earlier work this paper cites.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J Goodfellow, and Rob Fergus, Intriguing properties of neural networks. iclr, abs/1312.6199, 2014 , 2014
Original
2014
Earlier work this paper cites.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples , 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015
2015
Earlier work this paper cites.
Christos Thrampoulidis, Ehsan Abbasi, and Babak Hassibi, Precise high-dimensional error analysis of regularized m-estimators , 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), IEEE, 2015, pp. 410–417
2015
Earlier work this paper cites.
Christos Thrampoulidis, Samet Oymak, and Babak Hassibi, Regularized linear regression: A precise analysis of the estimation error , Conference on Learning Theory, 2015, pp. 1683–1709
2015
Earlier work this paper cites.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio, Adversarial machine learning at scale , arXiv preprint arXiv:1611.01236 (2016)
Original
2016
Earlier work this paper cites.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, Towards deep learning models resistant to adversarial attacks , arXiv preprint arXiv:1706.06083 (2017)
Original
2017
Earlier work this paper cites.
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal, Reconciling modern machine learning and the bias-variance trade-off , arXiv preprint arXiv:1812.11118 (2018)
Original
2018
Earlier work this paper cites.
Mikhail Belkin, Siyuan Ma, and Soumik Mandal, To understand deep learning we need to understand kernel learning , International Conference on Machine Learning, 2018, pp. 541–549
2018
Earlier work this paper cites.

Similar

Daniel Cullina, Arjun Nitin Bhagoji, and Prateek Mittal, Pac-learning in the presence of adversaries , Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada, 2018, pp. 228–239
2018
Cited alongside, same era.
Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S Schoenholz, Maithra Raghu, Martin Wattenberg, and Ian Goodfellow, Adversarial spheres , arXiv preprint arXiv:1801.02774 (2018)
Original
2018
Cited alongside, same era.
Justin Khim and Po-Ling Loh, Adversarial risk bounds for binary classification via function transformation , CoRR abs/1810.09519
Original
2018
Cited alongside, same era.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, Towards deep learning models resistant to adversarial attacks , 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, 2018

Then

Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan J Tibshirani, Surprises in high-dimensional ridgeless least squares interpolation , arXiv preprint arXiv:1903.08560 (2019)
Original
2019
Later among the works it cites.
Omar Montasser, Steve Hanneke, and Nathan Srebro, VC classes are adversarially robustly learnable, but only improperly , Conference on Learning Theory, COLT 2019, 25-28 June 2019, Phoenix, AZ, USA, 2019, pp. 2512–2530
2019
Later among the works it cites.
Song Mei and Andrea Montanari, The generalization error of random features regression: Precise asymptotics and double descent curve , arXiv preprint arXiv:1908.05355 (2019)
Original
2019
Later among the works it cites.
Andrea Montanari, Feng Ruan, Youngtak Sohn, and Jun Yan, The generalization error of max-margin linear classifiers: High-dimensional asymptotics in the overparametrized regime , arXiv preprint arXiv:1911.01544 (2019)
Original
2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2020

Precise Tradeoffs in Adversarial Training for Linear Regression

Javanmard, Adel, Soltanolkotabi, Mahdi, Hassani, Hamed

Understand

Despite breakthrough performance, modern learning models are known to be highly vulnerable to small adversarial perturbations in their inputs.

While a wide variety of recent \emph{adversarial training} methods have been effective at improving robustness to perturbed inputs (robust accuracy), often this benefit is accompanied by a decrease in accuracy on benign inputs (standard accuracy), leading to a tradeoff between often competing objectives.
Complicating matters further, recent empirical evidence suggest that a variety of other factors (size and quality of training data, model size, etc.) affect this tradeoff in somewhat surprising ways.
In this paper we provide a precise and comprehensive understanding of the role of adversarial training in the context of linear regression with Gaussian features.

Built on

Maurice Sion et al., On general minimax theorems. , Pacific Journal of mathematics 8
1958
Earlier work this paper cites.
Yehoram Gordon, On milman’s inequality and random subspaces which escape through a mesh in ℝ n \mathbb{R}^{n} , Geometric aspects of functional analysis, Springer, 1988, pp. 84–106
1988
Earlier work this paper cites.
Friedrich Liese and Klaus-J. Miescke, Statistical decision theory: Estimation, testing, and selection , Springer Science & Business Media, 2008
2008
Earlier work this paper cites.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli, Evasion attacks against machine learning at test time , Joint European conference on machine learning and knowledge discovery in databases, Springer, 2013, pp. 387–402
2013
Earlier work this paper cites.
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J Goodfellow, and Rob Fergus, Intriguing properties of neural networks. iclr, abs/1312.6199, 2014 , 2014
Original
2014
Earlier work this paper cites.
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples , 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015
2015
Earlier work this paper cites.
Christos Thrampoulidis, Ehsan Abbasi, and Babak Hassibi, Precise high-dimensional error analysis of regularized m-estimators , 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), IEEE, 2015, pp. 410–417
2015
Earlier work this paper cites.
Christos Thrampoulidis, Samet Oymak, and Babak Hassibi, Regularized linear regression: A precise analysis of the estimation error , Conference on Learning Theory, 2015, pp. 1683–1709
2015
Earlier work this paper cites.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio, Adversarial machine learning at scale , arXiv preprint arXiv:1611.01236 (2016)
Original
2016
Earlier work this paper cites.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, Towards deep learning models resistant to adversarial attacks , arXiv preprint arXiv:1706.06083 (2017)
Original
2017
Earlier work this paper cites.
Mikhail Belkin, Daniel Hsu, Siyuan Ma, and Soumik Mandal, Reconciling modern machine learning and the bias-variance trade-off , arXiv preprint arXiv:1812.11118 (2018)
Original
2018
Earlier work this paper cites.
Mikhail Belkin, Siyuan Ma, and Soumik Mandal, To understand deep learning we need to understand kernel learning , International Conference on Machine Learning, 2018, pp. 541–549
2018
Earlier work this paper cites.

Similar

Daniel Cullina, Arjun Nitin Bhagoji, and Prateek Mittal, Pac-learning in the presence of adversaries , Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada, 2018, pp. 228–239
2018
Cited alongside, same era.
Justin Gilmer, Luke Metz, Fartash Faghri, Samuel S Schoenholz, Maithra Raghu, Martin Wattenberg, and Ian Goodfellow, Adversarial spheres , arXiv preprint arXiv:1801.02774 (2018)
Original
2018
Cited alongside, same era.
Justin Khim and Po-Ling Loh, Adversarial risk bounds for binary classification via function transformation , CoRR abs/1810.09519
Original
2018
Cited alongside, same era.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu, Towards deep learning models resistant to adversarial attacks , 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, 2018

Then

Trevor Hastie, Andrea Montanari, Saharon Rosset, and Ryan J Tibshirani, Surprises in high-dimensional ridgeless least squares interpolation , arXiv preprint arXiv:1903.08560 (2019)
Original
2019
Later among the works it cites.
Omar Montasser, Steve Hanneke, and Nathan Srebro, VC classes are adversarially robustly learnable, but only improperly , Conference on Learning Theory, COLT 2019, 25-28 June 2019, Phoenix, AZ, USA, 2019, pp. 2512–2530
2019
Later among the works it cites.
Song Mei and Andrea Montanari, The generalization error of random features regression: Precise asymptotics and double descent curve , arXiv preprint arXiv:1908.05355 (2019)
Original
2019
Later among the works it cites.
Andrea Montanari, Feng Ruan, Youngtak Sohn, and Jun Yan, The generalization error of max-margin linear classifiers: High-dimensional asymptotics in the overparametrized regime , arXiv preprint arXiv:1911.01544 (2019)
Original
2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

2018

Cited alongside, same era.

Aditi Raghunathan, Jacob Steinhardt, and Percy Liang, Certified defenses against adversarial examples , 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings, 2018

2018

Cited alongside, same era.

Ludwig Schmidt, Shibani Santurkar, Dimitris Tsipras, Kunal Talwar, and Aleksander Madry, Adversarially robust generalization requires more data , Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, 3-8 December 2018, Montréal, Canada, 2018, pp. 5019–5031

2018

Cited alongside, same era.

Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry, Robustness may be at odds with accuracy , arXiv preprint arXiv:1805.12152 (2018)

Original

2018

Cited alongside, same era.

Eric Wong and J. Zico Kolter, Provable defenses against adversarial examples via the convex outer adversarial polytope , Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018, 2018, pp. 5283–5292

2018

Cited alongside, same era.

Sébastien Bubeck, Yin Tat Lee, Eric Price, and Ilya P. Razenshteyn, Adversarial examples from computational constraints , Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA, 2019, pp. 831–840

2019

Cited alongside, same era.

Zeyu Deng, Abla Kammoun, and Christos Thrampoulidis, A model of double descent for high-dimensional binary linear classification , arXiv preprint arXiv:1911.05822 (2019)

Original

2019

Cited alongside, same era.

Ruiqi Gao, Tianle Cai, Haochuan Li, Cho-Jui Hsieh, Liwei Wang, and Jason D. Lee, Convergence of adversarial training in overparametrized neural networks , Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada, 2019, pp. 13009–13020

2019

Cited alongside, same era.

Preetum Nakkiran, Adversarial robustness may be at odds with simplicity , arXiv preprint arXiv:1901.00532 (2019)

Original

2019