Efficient Domain Adaptation for Speech Foundation Models

Understand

Foundation models (FMs), that are trained on broad data at scale and are adaptable to a wide range of downstream tasks, have brought large interest in the research community.

• Benefiting from the diverse data sources such as different modalities, languages and application domains, foundation models have demonstrated strong generalization and knowledge transfer capabilities.
• In this paper, we present a pioneering study towards building an efficient solution for FM-based speech recognition systems.
• We adopt the recently developed self-supervised BEST-RQ for pretraining, and propose the joint finetuning with both source and unsupervised target domain data using JUST Hydra.