Training Priors Predict Text-To-Image Model Performance

Understand

Text-to-image models can often generate some relations, i.e., "astronaut riding horse", but fail to generate other relations composed of the same basic parts, i.e., "horse riding astronaut".

• These failures are often taken as evidence that models rely on training priors rather than constructing novel images compositionally.
• This paper tests this intuition on the stablediffusion 2.1 text-to-image model.
• By looking at the subject-verb-object (SVO) triads that underlie these prompts (e.g., "astronaut", "ride", "horse"), we find that the more often an SVO triad appears in the training data, the better the model can generate an image aligned with that triad.