Mechanisms of Symbol Processing for In-Context Learning in Transformer Networks

Understand

Large Language Models (LLMs) have demonstrated impressive abilities in symbol processing through in-context learning (ICL).

• This success flies in the face of decades of critiques asserting that artificial neural networks cannot master abstract symbol manipulation.
• We seek to understand the mechanisms that can enable robust symbol processing in transformer networks, illuminating both the unanticipated success, and the significant limitations, of transformers in symbol processing.
• Borrowing insights from symbolic AI and cognitive science on the power of Production System architectures, we develop a high-level Production System Language, PSL, that allows us to write symbolic programs to do complex, abstract symbol processing, and create compilers that precisely implement PSL programs in transformer networks which are, by construction, 100% mechanistically interpretable.