Human alignment of neural network representations

Understand

Today's computer vision models achieve human or near-human level performance across a wide variety of vision tasks.

• However, their architectures, data, and learning algorithms differ in numerous ways from those that give rise to human vision.
• In this paper, we investigate the factors that affect the alignment between the representations learned by neural networks and human mental representations inferred from behavioral responses.
• We find that model scale and architecture have essentially no effect on the alignment with human behavioral responses, whereas the training dataset and objective function both have a much larger impact.