LLMs for Relational Reasoning: How Far are We?

Understand

Large language models (LLMs) have revolutionized many areas (e.g.

• natural language processing, software engineering, etc.) by achieving state-of-the-art performance on extensive downstream tasks.
• Aiming to achieve robust and general artificial intelligence, there has been a surge of interest in investigating the reasoning ability of the LLMs.
• Whereas the textual and numerical reasoning benchmarks adopted by previous works are rather shallow and simple, it is hard to conclude that the LLMs possess strong reasoning ability by merely achieving positive results on these benchmarks.