IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models

Understand

The field of vision-and-language (VL) understanding has made unprecedented progress with end-to-end large pre-trained VL models (VLMs).

• However, they still fall short in zero-shot reasoning tasks that require multi-step inferencing.
• To achieve this goal, previous works resort to a divide-and-conquer pipeline.
• In this paper, we argue that previous efforts have several inherent shortcomings: 1) They rely on domain-specific sub-question decomposing models.