Fetching the paper…
Reading the bibliography…
State space models (SSM) have recently been shown to be very effective as a deep learning layer as a promising alternative to sequence models such as RNNs, CNNs, or Transformers.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber · 1997
Earlier work this paper cites.
Structured matrices and polynomials: unified superfast algorithms
Victor Pan · 2001
Earlier work this paper cites.
A first course in numerical analysis
Anthony Ralston and Philip Rabinowitz · 2001
Earlier work this paper cites.
Batch normalization: Accelerating deep network training by reducing internal covariate shift
Sergey Ioffe and Christian Szegedy · 2015
Earlier work this paper cites.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton · 2016
Earlier work this paper cites.
Language modeling with gated convolutional networks
Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier · 2017
Earlier work this paper cites.
Learning longer-term dependencies in RNNs with auxiliary losses
Trieu H Trinh, Andrew M Dai, Minh-Thang Luong, and Quoc V Le · 2018
Earlier work this paper cites.
Trellis networks for sequence modeling
Shaojie Bai, J Zico Kolter, and Vladlen Koltun · 2019
Earlier work this paper cites.
Legendre memory units: Continuous-time representation in recurrent neural networks
Aaron Voelker, Ivana Kajić, and Chris Eliasmith · 2019
Cited alongside, same era.
Neural controlled differential equations for irregular time series
Patrick Kidger, James Morrill, James Foster, and Terry Lyons · 2020
Cited alongside, same era.
Glu variants improve transformer
Noam Shazeer · 2020
Cited alongside, same era.
Lipschitz recurrent neural networks
N Benjamin Erichson, Omri Azencot, Alejandro Queiruga, Liam Hodgkinson, and Michael W Mahoney · 2021
Cited alongside, same era.
Combining recurrent, convolutional, and continuous-time models with the structured learnable linear state space layer
Albert Gu, Isys Johnson, Karan Goel, Khaled Saab, Tri Dao, Atri Rudra, and Christopher Ré · 2021
Cited alongside, same era.
Ckconv: Continuous kernel convolution for sequential data
David W Romero, Anna Kuzina, Erik J Bekkers, Jakub M Tomczak, and Mark Hoogendoorn · 2021
Later among the works it cites.
Unicornn: A recurrent model for learning very long time dependencies
T Konstantin Rusch and Siddhartha Mishra · 2021
Later among the works it cites.
Time series extrinsic regression
Chang Wei Tan, Christoph Bergmeir, Francois Petitjean, and Geoffrey I Webb · 2021
Later among the works it cites.
Long range arena : A benchmark for efficient transformers
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler · 2021
Later among the works it cites.
It’s raw! audio generation with state-space models
Karan Goel, Albert Gu, Chris Donahue, and Christopher Ré · 2022
Closest in time.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Neural rough differential equations for long time series
James Morrill, Cristopher Salvi, Patrick Kidger, James Foster, and Terry Lyons · 2021
Cited alongside, same era.
In-depth benchmarking of deep neural network architectures for ecg diagnosis
Naoki Nonaka and Jun Seita · 2021
Cited alongside, same era.
Hippo: Recurrent memory with optimal polynomial projections
Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré
Cited in the paper.
Improving the gating mechanism of recurrent neural networks
Albert Gu, Caglar Gulcehre, Tom Le Paine, Matt Hoffman, and Razvan Pascanu
Cited in the paper.
Efficiently modeling long sequences with structured state spaces
Albert Gu, Karan Goel, and Christopher Ré
Cited in the paper.
How to train your hippo: State space models with generalized basis projections
Albert Gu, Isys Johnson, Aman Timalsina, Atri Rudra, and Christopher Ré
Cited in the paper.
Ankit Gupta · 2022
Closest in time.
Flexconv: Continuous kernel convolutions with differentiable kernel sizes
David W Romero, Robert-Jan Bruintjes, Jakub M Tomczak, Erik J Bekkers, Mark Hoogendoorn, and Jan C van Gemert · 2022
Closest in time.