Empowering LLMs with Logical Reasoning: A Comprehensive Survey

Understand

Large language models (LLMs) have achieved remarkable successes on various tasks.

• However, recent studies have found that there are still significant challenges to the logical reasoning abilities of LLMs, which can be categorized into the following two aspects: (1) Logical question answering: LLMs often fail to generate the correct answer within a complex logical problem which requires sophisticated deductive, inductive or abductive reasoning given a collection of premises.
• (2) Logical consistency: LLMs are prone to producing responses contradicting themselves across different questions.
• For example, a state-of-the-art question-answering LLM Macaw, answers Yes to both questions Is a magpie a bird? and Does a bird have wings? but answers No to Does a magpie have wings?.