OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows

Understand

We introduce OmniFlow, a novel generative model designed for any-to-any generation tasks such as text-to-image, text-to-audio, and audio-to-image synthesis.

• OmniFlow advances the rectified flow (RF) framework used in text-to-image models to handle the joint distribution of multiple modalities.
• It outperforms previous any-to-any models on a wide range of tasks, such as text-to-image and text-to-audio synthesis.
• Our work offers three key contributions: First, we extend RF to a multi-modal setting and introduce a novel guidance mechanism, enabling users to flexibly control the alignment between different modalities in the generated outputs.