MiniGPT-5: Interleaved Vision-and-Language Generation via Generative Vokens

Understand

The effectiveness of Multimodal Large Language Models (MLLMs) demonstrates a profound capability in multimodal understanding.

• However, the simultaneous generation of images with coherent texts is still underdeveloped.
• Addressing this, we introduce a novel interleaved vision-and-language generation method, centered around the concept of ``generative vokens".
• These vokens serve as pivotal elements contributing to coherent image-text outputs.