MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response

Understand

Large Language Models (LLMs) have shown immense potential in multimodal applications, yet the convergence of textual and musical domains remains not well-explored.

• To address this gap, we present MusiLingo, a novel system for music caption generation and music-related query responses.
• MusiLingo employs a single projection layer to align music representations from the pre-trained frozen music audio model MERT with a frozen LLM, bridging the gap between music audio and textual contexts.
• We train it on an extensive music caption dataset and fine-tune it with instructional data.