From Words to Music: A Study of Subword Tokenization Techniques in Symbolic Music Generation

Understand

Subword tokenization has been widely successful in text-based natural language processing (NLP) tasks with Transformer-based models.

• As Transformer models become increasingly popular in symbolic music-related studies, it is imperative to investigate the efficacy of subword tokenization in the symbolic music domain.
• In this paper, we explore subword tokenization techniques, such as byte-pair encoding (BPE), in symbolic music generation and its impact on the overall structure of generated songs.
• Our experiments are based on three types of MIDI datasets: single track-melody only, multi-track with a single instrument, and multi-track and multi-instrument.