Recursion of Thought: A Divide-and-Conquer Approach to Multi-Context Reasoning with Language Models

Understand

Generating intermediate steps, or Chain of Thought (CoT), is an effective way to significantly improve language models' (LM) multi-step reasoning capability.

• However, the CoT lengths can grow rapidly with the problem complexity, easily exceeding the maximum context size.
• Instead of increasing the context limit, which has already been heavily investigated, we explore an orthogonal direction: making LMs divide a problem into multiple contexts.
• We propose a new inference framework, called Recursion of Thought (RoT), which introduces several special tokens that the models can output to trigger context-related operations.