Output Scouting: Auditing Large Language Models for Catastrophic Responses

Understand

Recent high profile incidents in which the use of Large Language Models (LLMs) resulted in significant harm to individuals have brought about a growing interest in AI safety.

• One reason LLM safety issues occur is that models often have at least some non-zero probability of producing harmful outputs.
• In this work, we explore the following scenario: imagine an AI safety auditor is searching for catastrophic responses from an LLM (e.g.
• a "yes" responses to "can I fire an employee for being pregnant?"), and is able to query the model a limited number times (e.g.