Human Attention in Fine-grained Classification

Understand

The way humans attend to, process and classify a given image has the potential to vastly benefit the performance of deep learning models.

• Exploiting where humans are focusing can rectify models when they are deviating from essential features for correct decisions.
• To validate that human attention contains valuable information for decision-making processes such as fine-grained classification, we compare human attention and model explanations in discovering important features.
• Towards this goal, we collect human gaze data for the fine-grained classification dataset CUB and build a dataset named CUB-GHA (Gaze-based Human Attention).