Mastering Symbolic Operations: Augmenting Language Models with Compiled Neural Networks

Understand

Language models' (LMs) proficiency in handling deterministic symbolic reasoning and rule-based tasks remains limited due to their dependency implicit learning on textual data.

• To endow LMs with genuine rule comprehension abilities, we propose "Neural Comprehension" - a framework that synergistically integrates compiled neural networks (CoNNs) into the standard transformer architecture.
• CoNNs are neural modules designed to explicitly encode rules through artificially generated attention weights.
• By incorporating CoNN modules, the Neural Comprehension framework enables LMs to accurately and robustly execute rule-intensive symbolic tasks.