Towards Artwork Explanation in Large-scale Vision Language Models

Understand

Large-scale Vision-Language Models (LVLMs) output text from images and instructions, demonstrating capabilities in text generation and comprehension.

• However, it has not been clarified to what extent LVLMs possess the ability to understand the knowledge necessary for explaining images, the complex relationships between various pieces of knowledge, and how they integrate these understandings into their explanations.
• To address this issue, we propose a new task: the artwork explanation generation task, along with its evaluation dataset and metrics for quantitatively assessing the understanding and utilization of knowledge about artworks.
• This task is apt for image description based on the premise that LVLMs are expected to have pre-existing knowledge of artworks, which are often subjects of wide recognition and documented information.