Text2midi: Generating Symbolic Music from Captions

Understand

This paper introduces text2midi, an end-to-end model to generate MIDI files from textual descriptions.

• Leveraging the growing popularity of multimodal generative approaches, text2midi capitalizes on the extensive availability of textual data and the success of large language models (LLMs).
• Our end-to-end system harnesses the power of LLMs to generate symbolic music in the form of MIDI files.
• Specifically, we utilize a pretrained LLM encoder to process captions, which then condition an autoregressive transformer decoder to produce MIDI sequences that accurately reflect the provided descriptions.