Speech ReaLLM -- Real-time Streaming Speech Recognition with Multimodal LLMs by Teaching the Flow of Time

N. Arivazhagan, C. Cherry, W. Macherey, C. Chiu, S. Yavuz, R. Pang, W. Li, and C. Raffel, “Monotonic infinite lookback attention for simultaneous machine translation,” CoRR , vol. abs/1906.05218, 2019. [Online]. Available: http://arxiv.org/abs/1906.05218

Original

1906

Earlier work this paper cites.

A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in Proceedings of ICML , 2006, pp. 369–376

2006

Earlier work this paper cites.

A. Graves, “Sequence transduction with recurrent neural networks,” CoRR , vol. abs/1211.3711, 2012. [Online]. Available: http://arxiv.org/abs/1211.3711

Original

2012

Earlier work this paper cites.

V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in Proceedings of ICASSP , 2015

2015

Earlier work this paper cites.

R. Prabhavalkar, K. Rao, T. N. Sainath, and Others, “A comparison of sequence-to-sequence models for speech recognition,” in Proceedings of Interspeech , 2017

2017

Earlier work this paper cites.

C. Raffel, M. Luong, P. J. Liu, R. J. Weiss, and D. Eck, “Online and linear-time attention by enforcing monotonic alignments,” CoRR , vol. abs/1704.00784, 2017. [Online]. Available: http://arxiv.org/abs/1704.00784

Original

2017

Earlier work this paper cites.

C.-C. Chiu* and C. Raffel*, “Monotonic chunkwise attention,” in International Conference on Learning Representations , 2018. [Online]. Available: https://openreview.net/forum?id=Hko85plCW

2018

Earlier work this paper cites.

D. S. Park, W. Chan, Y. Zhang, and Others, “SpecAugment: A simple data augmentation method for automatic speech recognition,” arXiv preprint arXiv:1904.08779 , 2019

Original

2019

Cited alongside, same era.

A. Gulati, J. Qin, C.-C. Chiu, and Others, “Conformer: Convolution-augmented transformer for speech recognition,” arXiv preprint arXiv:2005.08100 , 2020

Original

2020

Cited alongside, same era.

Y. Zhang, J. Qin, D. S. Park, and Others, “Pushing the limits of Semi-Supervised learning for automatic speech recognition,” arXiv preprint arXiv:2010.10504 , 2020

Original

2020

Cited alongside, same era.

Y. Shi, Y. Wang, C. Wu, and Others, “Emformer: Efficient memory transformer based acoustic model for low latency streaming speech recognition,” in Proceedings of ICASSP , 2021

2021

Cited alongside, same era.

J. Wu, Y. Gaur, Z. Chen, and Others, “On decoder-only architecture for speech-to-text and large language model integration,” in Proceedings of ASRU , 2023, pp. 1–8

2023

Later among the works it cites.

Y. Fathullah, C. Wu, E. Lakomkin, J. Jia, Y. Shangguan, K. Li, J. Guo, W. Xiong, J. Mahadeokar, O. Kalinli et al. , “Prompting large language models with speech recognition abilities,” arXiv preprint arXiv:2307.11795 , 2023

Original

2023

Later among the works it cites.

A. Radford, J. W. Kim, T. Xu, and Others, “Robust speech recognition via Large-Scale weak supervision,” in Proceedings of ICML , 2023

2023

Later among the works it cites.

L. Barrault et al. , “Seamlessm4t: Massively multilingual & multimodal machine translation,” ArXiv , 2023

2023

Later among the works it cites.

X. Ma, A. Sun, S. Ouyang, H. Inaguma, and P. Tomasello, “Efficient monotonic multihead attention,” 2023