Fetching the paper…
Reading the bibliography…
We introduce Centaurus, a class of networks composed of generalized state-space model (SSM) blocks, where the SSM operations can be treated as tensor contractions during training.
The general theory of relativity
Albert Einstein · 1922
Earlier work this paper cites.
An algorithm for the machine calculation of complex fourier series
James W Cooley and John W Tukey · 1965
Earlier work this paper cites.
State-space models
James D Hamilton · 1994
Earlier work this paper cites.
Tensor decompositions and applications
Tamara G Kolda and Brett W Bader · 2009
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Time series analysis: forecasting and control
George EP Box, Gwilym M Jenkins, Gregory C Reinsel, and Greta M Ljung · 2015
Earlier work this paper cites.
Deep learning
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton · 2015
Earlier work this paper cites.
Tensorizing neural networks
Alexander Novikov, Dmitrii Podoprikhin, Anton Osokin, and Dmitry P Vetrov · 2015
Earlier work this paper cites.
Librispeech: an asr corpus based on public domain audio books
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur · 2015
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Earlier work this paper cites.
Mobilenets: Efficient convolutional neural networks for mobile vision applications
Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam · 2017
Earlier work this paper cites.
Attention is all you need
A Vaswani · 2017
Earlier work this paper cites.
Neural ordinary differential equations
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud · 2018
Earlier work this paper cites.
Opt \ \backslash _einsum-a python package for optimizing contraction order for einsum-like expressions
G Daniel, Johnnie Gray, et al · 2018
Earlier work this paper cites.
Mobilenetv2: Inverted residuals and linear bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen · 2018
Earlier work this paper cites.
Speech commands: A dataset for limited-vocabulary speech recognition
Pete Warden · 2018
Earlier work this paper cites.
Fully convolutional speech recognition
Neil Zeghidour, Qiantong Xu, Vitaliy Liptchinsky, Nicolas Usunier, Gabriel Synnaeve, and Ronan Collobert · 2018
Earlier work this paper cites.
Sequence-to-sequence speech recognition with time-depth separable convolutions
Awni Hannun, Ann Lee, Qiantong Xu, and Ronan Collobert · 2019
Earlier work this paper cites.
Tensor networks for complex quantum systems
Román Orús · 2019
Cited alongside, same era.
Legendre memory units: Continuous-time representation in recurrent neural networks
Aaron Voelker, Ivana Kajić, and Chris Eliasmith · 2019
Cited alongside, same era.
wav2vec 2.0: A framework for self-supervised learning of speech representations
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli · 2020
Cited alongside, same era.
Real time speech enhancement in the waveform domain
Alexandre Defossez, Gabriel Synnaeve, and Yossi Adi · 2020
Cited alongside, same era.
Hippo: Recurrent memory with optimal polynomial projections
Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré · 2020
Cited alongside, same era.
Conformer: Convolution-augmented transformer for speech recognition
Frcrn: Boosting feature representation using frequency recurrence for monaural speech enhancement
Shengkui Zhao, Bin Ma, Karn N Watcharasupat, and Woon-Seng Gan · 2022
Later among the works it cites.
Mamba: Linear-time sequence modeling with selective state spaces
Albert Gu and Tri Dao · 2023
Later among the works it cites.
Parallelization of an ubiquitous sequential computation
Franz A Heinsen · 2023
Later among the works it cites.
Gateloop: Fully data-controlled linear recurrence for sequence modeling
Tobias Katsch · 2023
Later among the works it cites.
Structured state space decoder for speech recognition and synthesis
Koichi Miyazaki, Masato Murata, and Tomoki Koriyama · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, et al · 2020
Cited alongside, same era.
Wei Han, Zhengdong Zhang, Yu Zhang, Jiahui Yu, Chung-Cheng Chiu, James Qin, Anmol Gulati, Ruoming Pang, and Yonghui Wu · 2020
Cited alongside, same era.
Quartznet: Deep automatic speech recognition with 1d time-channel separable convolutions
Samuel Kriman, Stanislav Beliaev, Boris Ginsburg, Jocelyn Huang, Oleksii Kuchaiev, Vitaly Lavrukhin, Ryan Leary, Jason Li, and Yang Zhang · 2020
Cited alongside, same era.
A perceptually-motivated approach for low-complexity, real-time enhancement of fullband speech
Jean-Marc Valin, Umut Isik, Neerad Phansalkar, Ritwik Giri, Karim Helwani, and Arvindh Krishnaswamy · 2020
Cited alongside, same era.
Transformer transducer: A streamable speech recognition model with transformer encoders and rnn-t loss
Qian Zhang, Han Lu, Hasim Sak, Anshuman Tripathi, Erik McDermott, Stephen Koo, and Shankar Kumar · 2020
Cited alongside, same era.
Hyper-optimized tensor network contraction
Johnnie Gray and Stefanos Kourtis · 2021
Cited alongside, same era.
Efficiently modeling long sequences with structured state spaces
Albert Gu, Karan Goel, and Christopher Ré · 2021
Cited alongside, same era.
Later among the works it cites.
Hyena hierarchy: Towards larger convolutional language models
Michael Poli, Stefano Massaroli, Eric Nguyen, Daniel Y Fu, Tri Dao, Stephen Baccus, Yoshua Bengio, Stefano Ermon, and Christopher Ré · 2023
Later among the works it cites.
Deepfilternet: Perceptually motivated real-time speech enhancement
Hendrik Schröter, Tobias Rosenkranz, Andreas Maier, et al · 2023
Later among the works it cites.
Augmenting conformers with structured state space models for online speech recognition
Haozhe Shan, Albert Gu, Zhong Meng, Weiran Wang, Krzysztof Choromanski, and Tara Sainath · 2023
Later among the works it cites.
An investigation of incorporating mamba for speech enhancement
Rong Chao, Wen-Huang Cheng, Moreno La Quatra, Sabato Marco Siniscalchi, Chao-Han Huck Yang, Szu-Wei Fu, and Yu Tsao · 2024
Later among the works it cites.
Tri Dao and Albert Gu · 2024
Later among the works it cites.
Zamba: A compact 7b ssm hybrid model
Paolo Glorioso, Quentin Anthony, Yury Tokpanov, James Whittington, Jonathan Pilault, Adam Ibrahim, and Beren Millidge · 2024
Later among the works it cites.
Jamba: A hybrid transformer-mamba language model
Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, et al · 2024
Later among the works it cites.
Simba: Simplified mamba-based architecture for vision and multivariate time series
Badri N Patro and Vijay S Agneeswaran · 2024
Later among the works it cites.
Building temporal kernels with orthogonal polynomials
Yan Ru Pei and Olivier Coenen · 2024
Later among the works it cites.
Raw speech enhancement with deep state space modeling
Yan Ru Pei, Ritik Shrivastava, and FNU Sidharth · 2024
Later among the works it cites.
Samba: Simple hybrid state space models for efficient unlimited context language modeling
Liliang Ren, Yang Liu, Yadong Lu, Yelong Shen, Chen Liang, and Weizhu Chen · 2024
Later among the works it cites.
Mamba in speech: Towards an alternative to self-attention
Xiangyu Zhang, Qiquan Zhang, Hexin Liu, Tianyi Xiao, Xinyuan Qian, Beena Ahmed, Eliathamby Ambikairajah, Haizhou Li, and Julien Epps · 2024
Later among the works it cites.