M2-CTTS: End-to-End Multi-scale Multi-modal Conversational Text-to-Speech Synthesis

Understand

Conversational text-to-speech (TTS) aims to synthesize speech with proper prosody of reply based on the historical conversation.

• However, it is still a challenge to comprehensively model the conversation, and a majority of conversational TTS systems only focus on extracting global information and omit local prosody features, which contain important fine-grained information like keywords and emphasis.
• Moreover, it is insufficient to only consider the textual features, and acoustic features also contain various prosody information.
• Hence, we propose M2-CTTS, an end-to-end multi-scale multi-modal conversational text-to-speech system, aiming to comprehensively utilize historical conversation and enhance prosodic expression.