TroubleLLM: Align to Red Team Expert

Understand

Large Language Models (LLMs) become the start-of-the-art solutions for a variety of natural language tasks and are integrated into real-world applications.

• However, LLMs can be potentially harmful in manifesting undesirable safety issues like social biases and toxic content.
• It is imperative to assess its safety issues before deployment.
• However, the quality and diversity of test prompts generated by existing methods are still far from satisfactory.