FKA-Owl: Advancing Multimodal Fake News Detection through Knowledge-Augmented LVLMs

Understand

The massive generation of multimodal fake news involving both text and images exhibits substantial distribution discrepancies, prompting the need for generalized detectors.

• However, the insulated nature of training restricts the capability of classical detectors to obtain open-world facts.
• While Large Vision-Language Models (LVLMs) have encoded rich world knowledge, they are not inherently tailored for combating fake news and struggle to comprehend local forgery details.
• In this paper, we propose FKA-Owl, a novel framework that leverages forgery-specific knowledge to augment LVLMs, enabling them to reason about manipulations effectively.