AutoMOS: Learning a non-intrusive assessor of naturalness-of-speech

Understand

Developers of text-to-speech synthesizers (TTS) often make use of human raters to assess the quality of synthesized speech.

• We demonstrate that we can model human raters' mean opinion scores (MOS) of synthesized speech using a deep recurrent neural network whose inputs consist solely of a raw waveform.
• Our best models provide utterance-level estimates of MOS only moderately inferior to sampled human ratings, as shown by Pearson and Spearman correlations.
• When multiple utterances are scored and averaged, a scenario common in synthesizer quality assessment, AutoMOS achieves correlations approaching those of human raters.