NDVQ: Robust Neural Audio Codec with Normal Distribution-Based Vector Quantization

Understand

Built upon vector quantization (VQ), discrete audio codec models have achieved great success in audio compression and auto-regressive audio generation.

• However, existing models face substantial challenges in perceptual quality and signal distortion, especially when operating in extremely low bandwidth, rooted in the sensitivity of the VQ codebook to noise.
• This degradation poses significant challenges for several downstream tasks, such as codec-based speech synthesis.
• To address this issue, we propose a novel VQ method, Normal Distribution-based Vector Quantization (NDVQ), by introducing an explicit margin between the VQ codes via learning a variance.