Multi-resolution HuBERT: Multi-resolution Speech Self-Supervised Learning with Masked Unit Prediction

Understand

Existing Self-Supervised Learning (SSL) models for speech typically process speech signals at a fixed resolution of 20 milliseconds.

• This approach overlooks the varying informational content present at different resolutions in speech signals.
• In contrast, this paper aims to incorporate multi-resolution information into speech self-supervised representation learning.
• We introduce a SSL model that leverages a hierarchical Transformer architecture, complemented by HuBERT-style masked prediction objectives, to process speech at multiple resolutions.