Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implications

Understand

Recently, there have been breakthroughs in computer vision ("CV") models that are more generalizable with the advent of models such as CLIP and ALIGN.

• In this paper, we analyze CLIP and highlight some of the challenges such models pose.
• CLIP reduces the need for task specific training data, potentially opening up many niche tasks to automation.
• CLIP also allows its users to flexibly specify image classification classes in natural language, which we find can shift how biases manifest.