Using multi-task learning to improve the performance of acoustic-to-word and conventional hybrid models

Understand

Acoustic-to-word (A2W) models that allow direct mapping from acoustic signals to word sequences are an appealing approach to end-to-end automatic speech recognition due to their simplicity.

• However, prior works have shown that modelling A2W typically encounters issues of data sparsity that prevent training such a model directly.
• So far, pre-training initialization is the only approach proposed to deal with this issue.
• In this work, we propose to build a shared neural network and optimize A2W and conventional hybrid models in a multi-task manner.