Silence is Sweeter Than Speech: Self-Supervised Model Using Silence to Store Speaker Information

Understand

Self-Supervised Learning (SSL) has made great strides recently.

• SSL speech models achieve decent performance on a wide range of downstream tasks, suggesting that they extract different aspects of information from speech.
• However, how SSL models store various information in hidden representations without interfering is still poorly understood.
• Taking the recently successful SSL model, HuBERT, as an example, we explore how the SSL model processes and stores speaker information in the representation.