Streaming Multi-talker Speech Recognition with Joint Speaker Identification

Understand

In multi-talker scenarios such as meetings and conversations, speech processing systems are usually required to transcribe the audio as well as identify the speakers for downstream applications.

• Since overlapped speech is common in this case, conventional approaches usually address this problem in a cascaded fashion that involves speech separation, speech recognition and speaker identification that are trained independently.
• In this paper, we propose Streaming Unmixing, Recognition and Identification Transducer (SURIT) -- a new framework that deals with this problem in an end-to-end streaming fashion.
• SURIT employs the recurrent neural network transducer (RNN-T) as the backbone for both speech recognition and speaker identification.