On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning

Understand

Generating a Chain of Thought (CoT) has been shown to consistently improve large language model (LLM) performance on a wide range of NLP tasks.

• However, prior work has mainly focused on logical reasoning tasks (e.g.
• arithmetic, commonsense QA); it remains unclear whether improvements hold for more diverse types of reasoning, especially in socially situated contexts.
• Concretely, we perform a controlled evaluation of zero-shot CoT across two socially sensitive domains: harmful questions and stereotype benchmarks.