Differentiable Allophone Graphs for Language-Universal Speech Recognition

Understand

Building language-universal speech recognition systems entails producing phonological units of spoken sound that can be shared across languages.

• While speech annotations at the language-specific phoneme or surface levels are readily available, annotations at a universal phone level are relatively rare and difficult to produce.
• In this work, we present a general framework to derive phone-level supervision from only phonemic transcriptions and phone-to-phoneme mappings with learnable weights represented using weighted finite-state transducers, which we call differentiable allophone graphs.
• By training multilingually, we build a universal phone-based speech recognition model with interpretable probabilistic phone-to-phoneme mappings for each language.