Continuous Audio Language Models

Understand

Audio Language Models (ALM) have emerged as the dominant paradigm for speech and music generation by representing audio as sequences of discrete tokens.

• Yet, unlike text tokens, which are invertible, audio tokens are extracted from lossy codecs with a limited bitrate.
• As a consequence, increasing audio quality requires generating more tokens, which imposes a trade-off between fidelity and computational cost.
• We address this issue by studying Continuous Audio Language Models (CALM).