Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controls

Understand

We propose Polyffusion, a diffusion model that generates polyphonic music scores by regarding music as image-like piano roll representations.

• The model is capable of controllable music generation with two paradigms: internal control and external control.
• Internal control refers to the process in which users pre-define a part of the music and then let the model infill the rest, similar to the task of masked music generation (or music inpainting).
• External control conditions the model with external yet related information, such as chord, texture, or other features, via the cross-attention mechanism.