WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling

Understand

Language models have been effectively applied to modeling natural signals, such as images, video, speech, and audio.

• A crucial component of these models is the codec tokenizer, which compresses high-dimensional natural signals into lower-dimensional discrete tokens.
• In this paper, we introduce WavTokenizer, which offers several advantages over previous SOTA acoustic codec models in the audio domain: 1)extreme compression.
• By compressing the layers of quantizers and the temporal dimension of the discrete codec, one-second audio of 24kHz sampling rate requires only a single quantizer with 40 or 75 tokens.