A Comparative Investigation of Compositional Syntax and Semantics in DALL-E 2

Understand

In this study we compared how well DALL-E 2 visually represented the meaning of linguistic prompts also given to young children in comprehension tests.

• Sentences representing fundamental components of grammatical knowledge were selected from assessment tests used with several hundred English-speaking children aged 2-7 years for whom we had collected original item-level data.
• DALL-E 2 was given these prompts five times to generate 20 cartoons per item, for 9 adult judges to score.
• Results revealed no conditions in which DALL-E 2-generated images that matched the semantic accuracy of children, even at the youngest age (2 years).