Learning Transferable Visual Models From Natural Language Supervision

Understand

State-of-the-art computer vision systems are trained to predict a fixed set of predetermined object categories.

• This restricted form of supervision limits their generality and usability since additional labeled data is needed to specify any other visual concept.
• Learning directly from raw text about images is a promising alternative which leverages a much broader source of supervision.
• We demonstrate that the simple pre-training task of predicting which caption goes with which image is an efficient and scalable way to learn SOTA image representations from scratch on a dataset of 400 million (image, text) pairs collected from the internet.