ELITE: Enhanced Language-Image Toxicity Evaluation for Safety

Understand

Current Vision Language Models (VLMs) remain vulnerable to malicious prompts that induce harmful outputs.

• Existing safety benchmarks for VLMs primarily rely on automated evaluation methods, but these methods struggle to detect implicit harmful content or produce inaccurate evaluations.
• Therefore, we found that existing benchmarks have low levels of harmfulness, ambiguous data, and limited diversity in image-text pair combinations.
• To address these issues, we propose the ELITE benchmark, a high-quality safety evaluation benchmark for VLMs, underpinned by our enhanced evaluation method, the ELITE evaluator.