RuleBert: Teaching Soft Rules to Pre-trained Language Models

Understand

While pre-trained language models (PLMs) are the go-to solution to tackle many natural language processing problems, they are still very limited in their ability to capture and to use common-sense knowledge.

• In fact, even if information is available in the form of approximate (soft) logical rules, it is not clear how to transfer it to a PLM in order to improve its performance for deductive reasoning tasks.
• Here, we aim to bridge this gap by teaching PLMs how to reason with soft Horn rules.
• We introduce a classification task where, given facts and soft rules, the PLM should return a prediction with a probability for a given hypothesis.