Fetching the paper…
Reading the bibliography…
Imposing orthogonality on the layers of neural networks is known to facilitate the learning by limiting the exploding/vanishing of the gradient; decorrelate the features; improve the robustness.
Untersuchungen zu dynamischen neuronalen netzen
Sepp Hochreiter · 1991
Earlier work this paper cites.
Learning long-term dependencies with gradient descent is difficult
Yoshua Bengio, Patrice Simard, and Paolo Frasconi · 1994
Earlier work this paper cites.
Convolutional networks for images, speech, and time series
Yann LeCun and Yoshua Bengio · 1995
Earlier work this paper cites.
The geometry of algorithms with orthogonality constraints
Alan Edelman, Tomás A Arias, and Steven T Smith · 1998
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Robustness and generalization
Huan Xu and Shie Mannor · 2012
Earlier work this paper cites.
Auto-encoding variational bayes
D. P. Kingma and M. Welling · 2014
Earlier work this paper cites.
Intriguing properties of neural networks
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus · 2014
Earlier work this paper cites.
Fast and flexible convolutional sparse coding
Felix Heide, Wolfgang Heidrich, and Gordon Wetzstein · 2015
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Adam: A method for stochastic optimization
Diederik P. Kingma and Jimmy Ba · 2015
Earlier work this paper cites.
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
Anh Nguyen, Jason Yosinski, and Jeff Clune · 2015
Earlier work this paper cites.
Faster r-cnn: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun · 2015
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2015
Earlier work this paper cites.
Character-level convolutional networks for text classification
Xiang Zhang, Junbo Zhao, and Yann Lecun · 2015
Earlier work this paper cites.
Unitary evolution recurrent neural networks
Martin Arjovsky, Amar Shah, and Yoshua Bengio · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Efficient mobile implementation of a CNN-based object recognition system
Keiji Yanai, Ryosuke Tanno, and Koichi Okamoto · 2016
Earlier work this paper cites.
Towards principled methods for training generative adversarial networks
Martin Arjovsky and Léon Bottou · 2017
Cited alongside, same era.
Parseval networks: improving robustness to adversarial examples
Moustapha Cisse, Piotr Bojanowski, Edouard Grave, Yann Dauphin, and Nicolas Usunier · 2017
Cited alongside, same era.
Improved training of wasserstein gans
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville · 2017
Cited alongside, same era.
Fully convolutional networks for semantic segmentation
Evan Shelhamer, Jonathan Long, and Trevor Darrell · 2017
Cited alongside, same era.
Robust large margin deep neural networks
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues · 2017
Cited alongside, same era.
All you need is beyond a good init: Exploring better solution for training extremely deep convolutional neural networks with orthonormality and modulation
Approximated orthonormal normalisation in training neural networks
Guoqiang Zhang, Kenta Niwa, and W Bastiaan Kleijn · 2019
Later among the works it cites.
Separating the effects of batch normalization on cnn training speed and stability using classical adaptive filter theory
Elaina Chai, Mert Pilanci, and Boris Murmann · 2020
Later among the works it cites.
On the regularization of convolutional kernel tensors in neural networks
Pei-Chang Guo and Qiang Ye · 2020
Later among the works it cites.
Imagenette, 2020
Jeremy Howard · 2020
Later among the works it cites.
Controllable orthogonalization in training dnns
Lei Huang, Li Liu, Fan Zhu, Diwen Wan, Zehuan Yuan, Bo Li, and Ling Shao · 2020
Later among the works it cites.
Deep isometric learning for visual recognition
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Di Xie, Jiang Xiong, and Shiliang Pu · 2017
Cited alongside, same era.
Can we gain more from orthogonality regularizations in training deep networks?
Nitin Bansal, Xiaohan Chen, and Zhangyang Wang · 2018
Cited alongside, same era.
Generalizable adversarial training via spectral normalization
Farzan Farnia, Jesse Zhang, and David Tse · 2018
Cited alongside, same era.
Orthogonal weight normalization: Solution to optimization over multiple dependent stiefel manifolds in deep neural networks
Lei Huang, Xianglong Liu, Bo Lang, Adams Yu, Yongliang Wang, and Bo Li · 2018
Cited alongside, same era.
Towards deep learning models resistant to adversarial attacks
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu · 2018
Cited alongside, same era.
Spectral normalization for Generative Adversarial Networks
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida · 2018
Cited alongside, same era.
L2-nonexpansive neural networks
Haifeng Qian and Mark N Wegman · 2018
Cited alongside, same era.
Haozhi Qi, Chong You, Xiaolong Wang, Yi Ma, and Jitendra Malik · 2020
Later among the works it cites.
Orthogonal convolutional neural networks
Jiayun Wang, Yubei Chen, Rudrasis Chakraborty, and Stella X Yu · 2020
Later among the works it cites.
A closer look at accuracy vs. robustness
Yao-Yuan Yang, Cyrus Rashtchian, Hongyang Zhang, Russ R Salakhutdinov, and Kamalika Chaudhuri · 2020
Later among the works it cites.
Orthogonal convolutional neural networks for automatic sleep stage classification based on single-channel EEG
Junming Zhang, Ruxian Yao, Wengeng Ge, and Jinfeng Gao · 2020
Later among the works it cites.
Regularisation of neural networks by enforcing lipschitz continuity
Henry Gouk, Eibe Frank, Bernhard Pfahringer, and Michael J Cree · 2021
Closest in time.
The lipschitz constant of self-attention
Hyunjik Kim, George Papamakarios, and Andriy Mnih · 2021
Closest in time.
Orthogonal ensemble networks for biomedical image segmentation
Agostina J Larrazabal, César Martínez, Jose Dolz, and Enzo Ferrante · 2021
Closest in time.
Achieving robustness in classification using optimal transport with hinge regularization
Mathieu Serrurier, Franck Mamalet, Alberto González-Sanz, Thibaut Boissin, Jean-Michel Loubes, and Eustasio Del Barrio · 2021
Closest in time.
Skew orthogonal convolutions
Sahil Singla and Soheil Feizi · 2021
Closest in time.
Orthogonalizing convolutional layers with the Cayley Transform
Asher Trockman and J Zico Kolter · 2021
Closest in time.
Pay attention to your loss: understanding misconceptions about 1-lipschitz neural networks
Louis Béthune, Alberto González-Sanz, Franck Mamalet, and Mathieu Serrurier · 2022
Closest in time.
O-vit: Orthogonal vision transformer
Yanhong Fei, Yingjie Liu, Xian Wei, and Mingsong Chen · 2022
Closest in time.
Projunn: efficient method for training deep networks with unitary matrices
Bobak Kiani, Randall Balestriero, Yann LeCun, and Seth Lloyd · 2022
Closest in time.
Ortho-shot: Low displacement rank regularization with data augmentation for few-shot learning
Uche Osahor and Nasser M Nasrabadi · 2022
Closest in time.