Shrinking Bigfoot: Reducing wav2vec 2.0 footprint

Understand

Wav2vec 2.0 is a state-of-the-art speech recognition model which maps speech audio waveforms into latent representations.

• The largest version of wav2vec 2.0 contains 317 million parameters.
• Hence, the inference latency of wav2vec 2.0 will be a bottleneck in production, leading to high costs and a significant environmental footprint.
• To improve wav2vec's applicability to a production setting, we explore multiple model compression methods borrowed from the domain of large language models.