Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning

Understand

Recent advancements in multimodal large language models (MLLMs) have shown unprecedented capabilities in advancing various vision-language tasks.

• However, MLLMs face significant challenges with hallucinations, and misleading outputs that do not align with the input data.
• While existing efforts are paid to combat MLLM hallucinations, several pivotal challenges are still unsolved.
• First, while current approaches aggressively focus on addressing errors at the perception level, another important type at the cognition level requiring factual commonsense can be overlooked.