What Large Language Models Bring to Text-rich VQA?

Understand

Text-rich VQA, namely Visual Question Answering based on text recognition in the images, is a cross-modal task that requires both image comprehension and text recognition.

• In this work, we focus on investigating the advantages and bottlenecks of LLM-based approaches in addressing this problem.
• To address the above concern, we separate the vision and language modules, where we leverage external OCR models to recognize texts in the image and Large Language Models (LLMs) to answer the question given texts.
• The whole framework is training-free benefiting from the in-context ability of LLMs.