Fetching the paper…
Reading the bibliography…
In the quest for next-generation sequence modeling architectures, State Space Models (SSMs) have emerged as a potent alternative to transformers, particularly for their computational efficiency and suitability for dynamical systems.
Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluation, April 2020
Hao Wu, Patrick Judd, Xiaojie Zhang, Mikhail Isaev, and Paulius Micikevicius · 2004
Earlier work this paper cites.
Long Range Arena: A Benchmark for Efficient Transformers, November 2020
Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler · 2011
Earlier work this paper cites.
Recurrent Neural Networks With Limited Numerical Precision, February 2017
Joachim Ott, Zhouhan Lin, Ying Zhang, Shih-Chii Liu, and Yoshua Bengio · 2017
Earlier work this paper cites.
A Survey on Methods and Theories of Quantized Neural Networks, December 2018
Yunhui Guo · 2018
Earlier work this paper cites.
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio · 2018
Earlier work this paper cites.
Searching for mobilenetv3, 2019
Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, and Hartwig Adam · 2019
Earlier work this paper cites.
Legendre Memory Units: Continuous-Time Representation in Recurrent Neural Networks
Aaron Voelker, Ivana Kajić, and Chris Eliasmith · 2019
Earlier work this paper cites.
HiPPO: Recurrent Memory with Optimal Polynomial Projections
Albert Gu, Tri Dao, Stefano Ermon, Atri Rudra, and Christopher Ré · 2020
Earlier work this paper cites.
Pareto-Optimal Quantized ResNet Is Mostly 4-bit
AmirAli Abdolrashidi, Lisa Wang, Shivani Agrawal, Jonathan Malmaud, Oleg Rybakov, Chas Leichner, and Lukasz Lew · 2021
Earlier work this paper cites.
Hardware aware training for efficient keyword spotting on general purpose and specialized hardware
Peter Blouw, Gurshaant Malik, Benjamin Morcos, Aaron Voelker, and Chris Eliasmith · 2021
Earlier work this paper cites.
A Survey of Quantization Methods for Efficient Neural Network Inference, June 2021
Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer · 2021
Earlier work this paper cites.
On the quantization of recurrent neural networks, January 2021
Jian Li and Raziel Alvarez · 2021
Cited alongside, same era.
Deep double descent: where bigger models and more data hurt*
Preetum Nakkiran, Gal Kaplun, Yamini Bansal, Tristan Yang, Boaz Barak, and Ilya Sutskever · 2021
Cited alongside, same era.
Spiking reservoir computing for temporal edge intelligence on loihi
Ramashish Gaurav, Terrence C. Stewart, and Yang Cindy Yi · 2022
Cited alongside, same era.
Diagonal State Spaces are as Effective as Structured State Spaces, May 2022
Ankit Gupta, Albert Gu, and Jonathan Berant · 2022
Cited alongside, same era.
PokeBNN: A Binary Pursuit of Lightweight Accuracy, April 2022
Yichi Zhang, Zhiru Zhang, and Lukasz Lew · 2022
Cited alongside, same era.
Griffin: Mixing gated linear recurrences with local attention for efficient language models, 2024
Soham De, Samuel L. Smith, Anushan Fernando, Aleksandar Botev, George Cristian-Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, Arnaud Doucet, David Budden, Yee Whye Teh, Razvan Pascanu, Nando De Freitas, and Caglar Gulcehre · 2024
Closest in time.
Mamba: Linear-time sequence modeling with selective state spaces, 2024
Albert Gu and Tri Dao · 2024
Closest in time.
Jamba: A hybrid transformer-mamba language model, 2024
Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, Erez Safahi, Shaked Meirom, Yonatan Belinkov, Shai Shalev-Shwartz, Omri Abend, Raz Alon, Tomer Asida, Amir Bergman, Roman Glozman, Michael Gokhman, Avashalom Manevich, Nir Ratner, Noam Rozen, Erez Shwartz, Mor Zusman, and Yoav Shoham · 2024
Closest in time.
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits, February 2024
Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei · 2024
Closest in time.
Covariant spatio-temporal receptive fields for neuromorphic computing
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Sub-mw neuromorphic snn audio processing applications with rockpool and xylo
Hannah Bos and Dylan Muir · 2023
Cited alongside, same era.
4-bit Conformer with Native Quantization Aware Training for Speech Recognition, March 2023
Shaojin Ding, Phoenix Meadowlark, Yanzhang He, Lukasz Lew, Shivani Agrawal, and Oleg Rybakov · 2023
Cited alongside, same era.
Gaussian error linear units (gelus), 2023
Dan Hendrycks and Kevin Gimpel · 2023
Cited alongside, same era.
Resurrecting Recurrent Neural Networks for Long Sequences, March 2023
Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Caglar Gulcehre, Razvan Pascanu, and Soham De · 2023
Cited alongside, same era.
Simplified state space layers for sequence modeling
Jimmy T.H. Smith, Andrew Warrington, and Scott Linderman · 2023
Cited alongside, same era.
Binarized Neural Machine Translation, February 2023
Yichi Zhang, Ankush Garg, Yuan Cao, Łukasz Lew, Behrooz Ghorbani, Zhiru Zhang, and Orhan Firat · 2023
Cited alongside, same era.
Chaos as an interpretable benchmark for forecasting and data-driven modelling, 2023a
William Gilpin
Cited in the paper.
Jens Egholm Pedersen, Jörg Conradt, and Tony Lindeberg · 2024
Closest in time.
Efficient Video and Audio Processing with Loihi 2
Sumit Bam Shrestha, Jonathan Timcheck, Paxon Frady, Leobardo Campos-Macias, and Mike Davies · 2024
Closest in time.
A Survey on Transformer Compression, 2024
Yehui Tang, Yunhe Wang, Jianyuan Guo, Zhijun Tu, Kai Han, Hailin Hu, and Dacheng Tao · 2024
Closest in time.
Xindi Wang, Mahsa Salmani, Parsa Omidi, Xiangyu Ren, Mehdi Rezagholizadeh, and Armaghan Eshaghi · 2024
Closest in time.
Lu Yin, Ajay Jaiswal, Shiwei Liu, Souvik Kundu, and Zhangyang Wang · 2024
Closest in time.
A Survey on Efficient Inference for Large Language Models, 2024
Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, Shengen Yan, Guohao Dai, Xiao-Ping Zhang, Yuhan Dong, and Yu Wang · 2024
Closest in time.