UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and Voice Conversion

Understand

Text-to-speech (TTS) and voice conversion (VC) are two different tasks both aiming at generating high quality speaking voice according to different input modality.

• Due to their similarity, this paper proposes UnifySpeech, which brings TTS and VC into a unified framework for the first time.
• The model is based on the assumption that speech can be decoupled into three independent components: content information, speaker information, prosody information.
• Both TTS and VC can be regarded as mining these three parts of information from the input and completing the reconstruction of speech.