Mind with Eyes: from Language Reasoning to Multimodal Reasoning

Understand

Language models have recently advanced into the realm of reasoning, yet it is through multimodal reasoning that we can fully unlock the potential to achieve more comprehensive, human-like cognitive capabilities.

• This survey provides a systematic overview of the recent multimodal reasoning approaches, categorizing them into two levels: language-centric multimodal reasoning and collaborative multimodal reasoning.
• The former encompasses one-pass visual perception and active visual perception, where vision primarily serves a supporting role in language reasoning.
• The latter involves action generation and state update within reasoning process, enabling a more dynamic interaction between modalities.