Safe Inputs but Unsafe Output: Benchmarking Cross-modality Safety Alignment of Large Vision-Language Model

Understand

As Artificial General Intelligence (AGI) becomes increasingly integrated into various facets of human life, ensuring the safety and ethical alignment of such systems is paramount.

• Previous studies primarily focus on single-modality threats, which may not suffice given the integrated and complex nature of cross-modality interactions.
• We introduce a novel safety alignment challenge called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety alignment.
• Specifically, it considers cases where single modalities are safe independently but could potentially lead to unsafe or unethical outputs when combined.