USM-SCD: Multilingual Speaker Change Detection Based on Large Pretrained Foundation Models

Understand

We introduce a multilingual speaker change detection model (USM-SCD) that can simultaneously detect speaker turns and perform ASR for 96 languages.

• This model is adapted from a speech foundation model trained on a large quantity of supervised and unsupervised data, demonstrating the utility of fine-tuning from a large generic foundation model for a downstream task.
• We analyze the performance of this multilingual speaker change detection model through a series of ablation studies.
• We show that the USM-SCD model can achieve more than 75% average speaker change detection F1 score across a test set that consists of data from 96 languages.