Learning Language Representations with Logical Inductive Bias

Understand

Transformer architectures have achieved great success in solving natural language tasks, which learn strong language representations from large-scale unlabeled texts.

• In this paper, we seek to go further beyond and explore a new logical inductive bias for better language representation learning.
• Logic reasoning is known as a formal methodology to reach answers from given knowledge and facts.
• Inspired by such a view, we develop a novel neural architecture named FOLNet (First-Order Logic Network), to encode this new inductive bias.