Fetching the paper…
Reading the bibliography…
Large convolutional neural networks (CNN) can be difficult to train in the differentially private (DP) regime, since the optimization algorithms require a computationally expensive operation, known as the per-sample gradient clipping.
Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition
K. Fukushima and S. Miyake · 1982
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner · 1998
Earlier work this paper cites.
Calibrating noise to sensitivity in private data analysis
C. Dwork, F. McSherry, K. Nissim, and A. Smith · 2006
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
A. Krizhevsky, G. Hinton, et al · 2009
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
Playing atari with deep reinforcement learning
V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller · 2013
Earlier work this paper cites.
The algorithmic foundations of differential privacy
C. Dwork, A. Roth, et al · 2014
Earlier work this paper cites.
Generative adversarial nets
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio · 2014
Earlier work this paper cites.
One weird trick for parallelizing convolutional neural networks
A. Krizhevsky · 2014
Earlier work this paper cites.
Two-stream convolutional networks for action recognition in videos
K. Simonyan and A. Zisserman · 2014
Earlier work this paper cites.
Efficient per-example gradient computations
I. Goodfellow · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
K. Simonyan and A. Zisserman · 2015
Earlier work this paper cites.
Character-level convolutional networks for text classification
X. Zhang, J. Zhao, and Y. LeCun · 2015
Earlier work this paper cites.
Deep learning with differential privacy
M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
You only look once: Unified, real-time object detection
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi · 2016
Earlier work this paper cites.
Wide residual networks
S. Zagoruyko and N. Komodakis · 2016
Earlier work this paper cites.
Convolutional sequence to sequence learning
J. Gehring, M. Auli, D. Grangier, D. Yarats, and Y. N. Dauphin · 2017
Cited alongside, same era.
Cnn architectures for large-scale audio classification
S. Hershey, S. Chaudhuri, D. P. Ellis, J. F. Gemmeke, A. Jansen, R. C. Moore, M. Plakal, D. Platt, R. A. Saurous, B. Seybold, et al · 2017
Cited alongside, same era.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam · 2017
Cited alongside, same era.
Densely connected convolutional networks
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger · 2017
Cited alongside, same era.
Large kernel matters–improve semantic segmentation by global convolutional network
C. Peng, X. Zhang, G. Yu, G. Luo, and J. Sun · 2017
Cited alongside, same era.
Visformer: The vision-friendly transformer
Z. Chen, L. Xie, J. Niu, X. Liu, L. Wei, and Q. Tian · 2021
Later among the works it cites.
Convit: Improving vision transformers with soft convolutional inductive biases
S. d’Ascoli, H. Touvron, M. L. Leavitt, A. S. Morcos, G. Biroli, and L. Sagun · 2021
Later among the works it cites.
Rethinking spatial dimensions of vision transformers
B. Heo, S. Yun, D. Han, S. Chun, J. Choe, and S. J. Oh · 2021
Later among the works it cites.
Large language models can be strong differentially private learners
X. Li, F. Tramer, P. Liang, and T. Hashimoto · 2021
Later among the works it cites.
Tempered sigmoid activations for deep learning with differential privacy
N. Papernot, A. Thakurta, S. Song, S. Chien, and Ú. Erlingsson · 2021
Later among the works it cites.
Enabling fast differentially private sgd via just-in-time compilation and vectorization
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
S. Xie, R. Girshick, P. Dollár, Z. Tu, and K. He · 2017
Cited alongside, same era.
JAX: composable transformations of Python+NumPy programs, 2018
J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang · 2018
Cited alongside, same era.
Efficient per-example gradient computations in convolutional neural networks
G. Rochette, A. Manoel, and E. W. Tramel · 2019
Cited alongside, same era.
Pytorch image models
R. Wightman · 2019
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, et al · 2020
Cited alongside, same era.
Scaling up differentially private deep learning with fast per-example gradient clipping
J. Lee and D. Kifer · 2020
Cited alongside, same era.
Differentially private learning needs better features (or much more data)
F. Tramer and D. Boneh · 2020
Cited alongside, same era.
P. Subramani, N. Vadivelu, and G. Kamath · 2021
Later among the works it cites.
Training data-efficient image transformers & distillation through attention
H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. Jégou · 2021
Later among the works it cites.
Going deeper with image transformers
H. Touvron, M. Cord, A. Sablayrolles, G. Synnaeve, and H. Jégou · 2021
Later among the works it cites.
Opacus: User-friendly differential privacy library in pytorch
A. Yousefpour, I. Shilov, A. Sablayrolles, D. Testuggine, K. Prasad, M. Malek, J. Nguyen, S. Ghosh, A. Bharadwaj, J. Zhao, et al · 2021
Later among the works it cites.
Automatic clipping: Differentially private deep learning made easier and stronger
Z. Bu, Y.-X. Wang, S. Zha, and G. Karypis · 2022
Closest in time.
Unlocking high-accuracy differentially private image classification through scale
S. De, L. Berrada, J. Hayes, S. L. Smith, and B. Balle · 2022
Closest in time.
Scaling up your kernels to 31x31: Revisiting large kernel design in cnns
X. Ding, X. Zhang, J. Han, and G. Ding · 2022
Closest in time.
Differentially private training of residual networks with scale normalisation
H. Klause, A. Ziller, D. Rueckert, K. Hammernik, and G. Kaissis · 2022
Closest in time.
Toward training at imagenet scale with differential privacy
A. Kurakin, S. Chien, S. Song, R. Geambasu, A. Terzis, and A. Thakurta · 2022
Closest in time.
Large scale transfer learning for differentially private image classification
H. Mehta, A. Thakurta, A. Kurakin, and A. Cutkosky · 2022
Closest in time.
Scalablevit: Rethinking the context-oriented generalization of vision transformer
R. Yang, H. Ma, J. Wu, Y. Tang, X. Xiao, M. Zheng, and X. Li · 2022
Closest in time.
Nested hierarchical transformer: Towards accurate, data-efficient and interpretable visual understanding
Z. Zhang, H. Zhang, L. Zhao, T. Chen, S. Arik, and T. Pfister · 2022
Closest in time.