Preemptive Answer "Attacks" on Chain-of-Thought Reasoning

Understand

Large language models (LLMs) showcase impressive reasoning capabilities when coupled with Chain-of-Thought (CoT) prompting.

• However, the robustness of this approach warrants further investigation.
• In this paper, we introduce a novel scenario termed preemptive answers, where the LLM obtains an answer before engaging in reasoning.
• This situation can arise inadvertently or induced by malicious users by prompt injection attacks.