Improving Image Recognition by Retrieving from Web-Scale Image-Text Data

Understand

Retrieval augmented models are becoming increasingly popular for computer vision tasks after their recent success in NLP problems.

• The goal is to enhance the recognition capabilities of the model by retrieving similar examples for the visual input from an external memory set.
• In this work, we introduce an attention-based memory module, which learns the importance of each retrieved example from the memory.
• Compared to existing approaches, our method removes the influence of the irrelevant retrieved examples, and retains those that are beneficial to the input query.