Building a mixed-lingual neural TTS system with only monolingual data

Understand

When deploying a Chinese neural text-to-speech (TTS) synthesis system, one of the challenges is to synthesize Chinese utterances with English phrases or words embedded.

• This paper looks into the problem in the encoder-decoder framework when only monolingual data from a target speaker is available.
• Specifically, we view the problem from two aspects: speaker consistency within an utterance and naturalness.
• We start the investigation with an Average Voice Model which is built from multi-speaker monolingual data, i.e.