Learning Visual Prompts for Guiding the Attention of Vision Transformers

Understand

Visual prompting infuses visual information into the input image to adapt models toward specific predictions and tasks.

• Recently, manually crafted markers such as red circles are shown to guide the model to attend to a target region on the image.
• However, these markers only work on models trained with data containing those markers.
• Moreover, finding these prompts requires guesswork or prior knowledge of the domain on which the model is trained.