Supporting Human-AI Collaboration in Auditing LLMs with LLMs

Understand

Large language models are becoming increasingly pervasive and ubiquitous in society via deployment in sociotechnical systems.

• Yet these language models, be it for classification or generation, have been shown to be biased and behave irresponsibly, causing harm to people at scale.
• It is crucial to audit these language models rigorously.
• Existing auditing tools leverage either or both humans and AI to find failures.