Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learning

Understand

In this paper, we are tackling the proposal-free referring expression grounding task, aiming at localizing the target object according to a query sentence, without relying on off-the-shelf object proposals.

• Existing proposal-free methods employ a query-image matching branch to select the highest-score point in the image feature map as the target box center, with its width and height predicted by another branch.
• Such methods, however, fail to utilize the contextual relation between the target and reference objects, and lack interpretability on its reasoning procedure.
• To solve these problems, we propose an iterative shrinking mechanism to localize the target, where the shrinking direction is decided by a reinforcement learning agent, with all contents within the current image patch comprehensively considered.