LaViDa: A Large Diffusion Language Model for Multimodal Understanding

Understand

Modern Vision-Language Models (VLMs) can solve a wide range of tasks requiring visual reasoning.

• In real-world scenarios, desirable properties for VLMs include fast inference and controllable generation (e.g., constraining outputs to adhere to a desired format).
• However, existing autoregressive (AR) VLMs like LLaVA struggle in these aspects.
• Discrete diffusion models (DMs) offer a promising alternative, enabling parallel decoding for faster inference and bidirectional context for controllable generation through text-infilling.