Twitter-COMMs: Detecting Climate, COVID, and Military Multimodal Misinformation

Understand

Detecting out-of-context media, such as "mis-captioned" images on Twitter, is a relevant problem, especially in domains of high public significance.

• In this work we aim to develop defenses against such misinformation for the topics of Climate Change, COVID-19, and Military Vehicles.
• We first present a large-scale multimodal dataset with over 884k tweets relevant to these topics.
• Next, we propose a detection method, based on the state-of-the-art CLIP model, that leverages automatically generated hard image-text mismatches.