Fetching the paper…
Reading the bibliography…
Modern speech processing systems rely on self-attention.
A. Rousseau, P. Deléglise, Y. Esteve et al. , “Enhancing the ted-lium corpus with selected data for language modeling and more ted talks.” in LREC , 2014, pp. 3935–3939
2014
Earlier work this paper cites.
V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: an asr corpus based on public domain audio books,” in 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP) . IEEE, 2015, pp. 5206–5210
2015
Earlier work this paper cites.
H. Arasteh, V. Hosseinnezhad, V. Loia, A. Tommasetti, O. Troisi, M. Shafie-khah, and P. Siano, “Iot-based smart cities: A survey,” in 2016 IEEE 16th international conference on environment and electrical engineering (EEEIC) . IEEE, 2016, pp. 1–6
2016
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017
2017
Earlier work this paper cites.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems , 2017
2017
Earlier work this paper cites.
H. Bu, J. Du, X. Na, B. Wu, and H. Zheng, “Aishell-1: An open-source mandarin speech corpus and a speech recognition baseline,” in 2017 20th conference of the oriental chapter of the international coordinating committee on speech databases and speech I/O systems and assessment (O-COCOSDA) . IEEE, 2017, pp. 1–5
2017
Earlier work this paper cites.
A. B. Nassif, I. Shahin, I. Attili, M. Azzeh, and K. Shaalan, “Speech recognition using deep neural networks: A systematic review,” IEEE access , vol. 7, pp. 19 143–19 165, 2019
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
S. Karita, N. Chen, T. Hayashi, T. Hori, H. Inaguma, Z. Jiang, M. Someki, N. E. Y. Soplin, R. Yamamoto, X. Wang et al. , “A comparative study on transformer vs rnn in speech applications,” in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2019, pp. 449–456
2019
Earlier work this paper cites.
2019
Earlier work this paper cites.
A. Gulati, J. Qin, C.-C. Chiu, N. Parmar, Y. Zhang, J. Yu, W. Han, S. Wang, Z. Zhang, Y. Wu et al. , “Conformer: Convolution-augmented transformer for speech recognition,” Proc. Interspeech 2020 , pp. 5036–5040, 2020
2020
Earlier work this paper cites.
2020
Earlier work this paper cites.
2020
Cited alongside, same era.
M. Zaheer, G. Guruganesh, K. A. Dubey, J. Ainslie, C. Alberti, S. Ontanon, P. Pham, A. Ravula, Q. Wang, L. Yang et al. , “Big bird: Transformers for longer sequences,” Advances in neural information processing systems , vol. 33, pp. 17 283–17 297, 2020
2020
Cited alongside, same era.
W. Han, Z. Zhang, Y. Zhang, J. Yu, C.-C. Chiu, J. Qin, A. Gulati, R. Pang, and Y. Wu, “Contextnet: Improving convolutional neural networks for automatic speech recognition with global context,” Proc. Interspeech 2020 , pp. 3610–3614, 2020
2020
Cited alongside, same era.
R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Henretty, R. Morais, L. Saunders, F. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” in Proceedings of the 12th Language Resources and Evaluation Conference , 2020, pp. 4218–4222
J. Sakuma, T. Komatsu, and R. Scheibler, “Mlp-based architecture with variable length input for automatic speech recognition,” OpenReview , 2021
2021
Later among the works it cites.
P. Guo, F. Boyer, X. Chang, T. Hayashi, Y. Higuchi, H. Inaguma, N. Kamo, C. Li, D. Garcia-Romero, J. Shi et al. , “Recent developments on espnet toolkit boosted by conformer,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 5874–5878
2021
Later among the works it cites.
2021
Later among the works it cites.
2022
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
2020
Cited alongside, same era.
E. Bastianelli, A. Vanzo, P. Swietojanski, and V. Rieser, “Slurp: A spoken language understanding resource package,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2020, pp. 7252–7262
2020
Cited alongside, same era.
T. Parcollet and M. Ravanelli, “The Energy and Carbon Footprint of Training End-to-End Speech Recognizers,” in Proc. Interspeech 2021 , 2021, pp. 4583–4587
2021
Cited alongside, same era.
S. Zhang, E. Loweimi, P. Bell, and S. Renals, “On the usefulness of self-attention for automatic speech recognition with transformers,” in 2021 IEEE Spoken Language Technology Workshop (SLT) . IEEE, 2021, pp. 89–96
2021
Cited alongside, same era.
M. Burchi and V. Vielzeuf, “Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognition,” in 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) . IEEE, 2021, pp. 8–15
2021
Cited alongside, same era.
Y. Shi, Y. Wang, C. Wu, C.-F. Yeh, J. Chan, F. Zhang, D. Le, and M. Seltzer, “Emformer: Efficient memory transformer based acoustic model for low latency streaming speech recognition,” in ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2021, pp. 6783–6787
2021
Cited alongside, same era.
2021
Cited alongside, same era.
I. O. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Unterthiner, J. Yung, A. Steiner, D. Keysers, J. Uszkoreit et al. , “MLP-Mixer: An all-MLP architecture for vision,” in Advances in Neural Information Processing Systems , 2021
2021
Cited alongside, same era.
Y. Peng, S. Dalmia, I. Lane, and S. Watanabe, “Branchformer: Parallel MLP-attention architectures to capture local and global context for speech recognition and understanding,” in International Conference on Machine Learning . PMLR, 2022, pp. 17 627–17 643
2022
Later among the works it cites.
K. Shim, J. Choi, and W. Sung, “Understanding the role of self attention for efficient speech recognition,” in International Conference on Learning Representations , 2022
2022
Later among the works it cites.
Y. Tay, M. Dehghani, D. Bahri, and D. Metzler, “Efficient transformers: A survey,” ACM Computing Surveys , vol. 55, no. 6, pp. 1–28, 2022
2022
Later among the works it cites.
S. Kim, A. Gholami, A. Shaw, N. Lee, K. Mangalam, J. Malik, M. W. Mahoney, and K. Keutzer, “Squeezeformer: An efficient transformer for automatic speech recognition,” Advances in Neural Information Processing Systems , vol. 35, pp. 9361–9373, 2022
2022
Later among the works it cites.
L. Samarakoon and T.-Y. Leung, “Conformer-based speech recognition with linear nyström attention and rotary position embedding,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 8012–8016
2022
Later among the works it cites.
2022
Later among the works it cites.
F. Mai, A. Pannatier, F. Fehr, H. Chen, F. Marelli, F. Fleuret, and J. Henderson, “HyperMixer: An MLP-based low cost alternative to transformers,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Toronto, Canada: Association for Computational Linguistics, Jul. 2023, pp. 15 632–15 654. [Online]. Available: https://aclanthology.org/2023.acl-long.871
2023
Closest in time.