Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms

Understand

Modern vision models are trained on very large noisy datasets.

• While these models acquire strong capabilities, they may not follow the user's intent to output the desired results in certain aspects, e.g., visual aesthetic, preferred style, and responsibility.
• In this paper, we target the realm of visual aesthetics and aim to align vision models with human aesthetic standards in a retrieval system.
• Advanced retrieval systems usually adopt a cascade of aesthetic models as re-rankers or filters, which are limited to low-level features like saturation and perform poorly when stylistic, cultural or knowledge contexts are involved.