Solving Situation Puzzles with Large Language Model and External Reformulation

Understand

In recent years, large language models (LLMs) have shown an impressive ability to perform arithmetic and symbolic reasoning tasks.

• However, we found that LLMs (e.g., ChatGPT) cannot perform well on reasoning that requires multiple rounds of dialogue, especially when solving situation puzzles.
• Specifically, LLMs intend to ask very detailed questions focusing on a specific aspect or same/similar questions after several rounds of Q&As.
• To help LLMs get out of the above dilemma, we propose a novel external reformulation methodology, where the situation puzzle will be reformulated after several rounds of Q&A or when the LLMs raise an incorrect guess.