Fetching the paper…
Reading the bibliography…
We consider two biologically plausible structures, the Spiking Neural Network (SNN) and the self-attention mechanism.
Networks of spiking neurons: the third generation of neural network models
Wolfgang Maass · 1997
Earlier work this paper cites.
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei · 2009
Earlier work this paper cites.
Learning multiple layers of features from tiny images
Alex Krizhevsky · 2009
Earlier work this paper cites.
1.1 computing’s energy problem (and what we can do about it)
Mark Horowitz · 2014
Earlier work this paper cites.
A million spiking-neuron integrated circuit with a scalable communication network and interface
Paul A Merolla, John V Arthur, Rodrigo Alvarez-Icaza, Andrew S Cassidy, Jun Sawada, Filipp Akopyan, Bryan L Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, et al · 2014
Earlier work this paper cites.
Spiking deep convolutional neural networks for energy-efficient object recognition
Yongqiang Cao, Yang Chen, and Deepak Khosla · 2015
Earlier work this paper cites.
Spiking deep networks with lif neurons
Eric Hunsberger and Chris Eliasmith · 2015
Earlier work this paper cites.
Training deep spiking neural networks using backpropagation
Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer · 2016
Earlier work this paper cites.
A low power, fully event-based gesture recognition system
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, Jeff Kusnitz, Michael Debole, Steve Esser, Tobi Delbruck, Myron Flickner, and Dharmendra Modha · 2017
Earlier work this paper cites.
Cifar10-dvs: an event-stream dataset for object classification
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi · 2017
Earlier work this paper cites.
Conversion of continuous-valued deep networks to efficient event-driven networks for image classification
Bodo Rueckauer, Iulia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu · 2017
Earlier work this paper cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin · 2017
Earlier work this paper cites.
Slayer: Spike layer error reassignment in time
Sumit B Shrestha and Garrick Orchard · 2018
Earlier work this paper cites.
Spatio-temporal backpropagation for training high-performance spiking neural networks
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi · 2018
Earlier work this paper cites.
Surrogate gradient learning in spiking neural networks: Bringing the power of gradient-based optimization to spiking neural networks
Emre O Neftci, Hesham Mostafa, and Friedemann Zenke · 2019
Earlier work this paper cites.
Pytorch: An imperative style, high-performance deep learning library
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al · 2019
Earlier work this paper cites.
Towards spike-based machine intelligence with neuromorphic computing
Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda · 2019
Earlier work this paper cites.
Pytorch image models
Ross Wightman · 2019
Earlier work this paper cites.
Direct Training for Spiking Neural Networks: Faster, Larger, Better
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi · 2019
Earlier work this paper cites.
End-to-end object detection with transformers
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko · 2020
Earlier work this paper cites.
Rethinking attention with performers
Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al · 2020
Cited alongside, same era.
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al · 2020
Cited alongside, same era.
Spikingjelly
Wei Fang, Yanqi Chen, Jianhao Ding, Ding Chen, Zhaofei Yu, Huihui Zhou, Yonghong Tian, and other contributors · 2020
Cited alongside, same era.
Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network
Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy · 2020
Cited alongside, same era.
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
Jacques Kaiser, Hesham Mostafa, and Emre Neftci · 2020
Cited alongside, same era.
Optimizing Deeper Spiking Neural Networks for Dynamic Vision Sensing
Youngeun Kim and Priyadarshini Panda · 2021
Later among the works it cites.
Differentiable Spike: Rethinking Gradient-Descent for Training Spiking Neural Networks
Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu · 2021
Later among the works it cites.
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo · 2021
Later among the works it cites.
Spiking transformer networks: A rate coded approach for processing sequential data
Etienne Mueller, Viktor Studenyak, Daniel Auge, and Alois Knoll · 2021
Later among the works it cites.
Dynamicvit: Efficient vision transformers with dynamic token sparsification
Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh · 2021
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Transformers are rnns: Fast autoregressive transformers with linear attention
Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret · 2020
Cited alongside, same era.
Efficient Processing of Spatio-temporal Data Streams with Spiking Neural Networks
Alexander Kugele, Thomas Pfeil, Michael Pfeiffer, and Elisabetta Chicca · 2020
Cited alongside, same era.
Enabling spike-based backpropagation for training deep neural network architectures
Chankyu Lee, Syed Shakib Sarwar, Priyadarshini Panda, Gopalakrishnan Srinivasan, and Kaushik Roy · 2020
Cited alongside, same era.
Long short-term memory spiking networks and their applications
Ali Lotfi Rezaabad and Sriram Vishwanath · 2020
Cited alongside, same era.
Toward scalable, efficient, and accurate deep spiking neural networks with backward residual connections, stochastic softmax, and hybridization
Priyadarshini Panda, Sai Aparna Aketi, and Kaushik Roy · 2020
Cited alongside, same era.
Nitin Rathi and Kaushik Roy · 2020
Cited alongside, same era.
Nitin Rathi, Gopalakrishnan Srinivasan, Priyadarshini Panda, and Kaushik Roy · 2020
Cited alongside, same era.
Jeong-geun Song · 2021
Later among the works it cites.
Pyramid vision transformer: A versatile backbone for dense prediction without convolutions
Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao · 2021
Later among the works it cites.
LIAF-Net: Leaky Integrate and Analog Fire Network for Lightweight and Efficient Spatiotemporal Information Processing
Zhenzhi Wu, Hehui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang · 2021
Later among the works it cites.
Focal attention for long-range interactions in vision transformers
Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, and Jianfeng Gao · 2021
Later among the works it cites.
Temporal-wise attention spiking neural networks for event streams classification
Man Yao, Huanhuan Gao, Guangshe Zhao, Dingheng Wang, Yihan Lin, Zhaoxu Yang, and Guoqi Li · 2021
Later among the works it cites.
Accurate and efficient time-domain classification with adaptive spiking recurrent neural networks
Bojian Yin, Federico Corradi, and Sander M Bohté · 2021
Later among the works it cites.
Going Deeper With Directly-Trained Larger Spiking Neural Networks
Hanle Zheng, Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li · 2021
Later among the works it cites.
Brains and algorithms partially converge in natural language processing
Charlotte Caucheteux and Jean-Rémi King · 2022
Closest in time.
Neuromorphic data augmentation for training spiking neural networks
Yuhang Li, Youngeun Kim, Hyoungseob Park, Tamar Geller, and Priyadarshini Panda · 2022
Closest in time.
Qingyan Meng, Mingqing Xiao, Shen Yan, Yisen Wang, Zhouchen Lin, and Zhi-Quan Luo · 2022
Closest in time.
cosformer: Rethinking softmax in attention
Zhen Qin, Weixuan Sun, Hui Deng, Dongxu Li, Yunshen Wei, Baohong Lv, Junjie Yan, Lingpeng Kong, and Yiran Zhong · 2022
Closest in time.
Signed neuron with memory: Towards simple, accurate and high-efficient ann-snn conversion
Yuchen Wang, Malu Zhang, Yi Chen, and Hong Qu · 2022
Closest in time.
Relating transformers to models and neural representations of the hippocampal formation
James C. R. Whittington, Joseph Warren, and Tim E.J. Behrens · 2022
Closest in time.
Attention spiking neural networks
Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, and Guoqi Li · 2022
Closest in time.
Spiking graph convolutional networks
Zulun Zhu, Jiaying Peng, Jintang Li, Liang Chen, Qi Yu, and Siqiang Luo · 2022
Closest in time.