Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?

Understand

In this paper, we explain the inference logic of large language models (LLMs) as a set of symbolic concepts.

• Many recent studies have discovered that traditional DNNs usually encode sparse symbolic concepts.
• However, because an LLM has much more parameters than traditional DNNs, whether the LLM also encodes sparse symbolic concepts is still an open problem.
• Therefore, in this paper, we propose to disentangle the inference score of LLMs for dialogue tasks into a small number of symbolic concepts.