Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction

Understand

Humans can easily isolate a single speaker from a complex acoustic environment, a capability referred to as the "Cocktail Party Effect." However, replicating this ability has been a significant challenge in the field of target speaker extraction (TSE).

• Traditional TSE approaches predominantly rely on voiceprints, which raise privacy concerns and face issues related to the quality and availability of enrollment samples, as well as intra-speaker variability.
• To address these issues, this work introduces a novel text-guided TSE paradigm named LLM-TSE.
• In this paradigm, a state-of-the-art large language model, LLaMA 2, processes typed text input from users to extract semantic cues.