2021

Learning Symbolic Rules for Reasoning in Quasi-Natural Language

Yang, Kaiyu, Deng, Jia

Understand

Symbolic reasoning, rule-based symbol manipulation, is a hallmark of human intelligence.

However, rule-based systems have had limited success competing with learning-based systems outside formalized domains such as automated theorem proving.
We hypothesize that this is due to the manual construction of rules in past attempts.
In this work, we ask how we can build a rule-based system that can reason with natural language input but without the manual construction of rules.

Built on

A note on inductive generalization
Gordon Plotkin · 1970
Earlier work this paper cites.
Automatic methods of inductive inference
Gordon Plotkin · 1972
Earlier work this paper cites.
Tensor product variable binding and the representation of symbolic structures in connectionist systems
Paul Smolensky · 1990
Earlier work this paper cites.
Inductive logic programming
Stephen Muggleton · 1991
Earlier work this paper cites.
Taxonomic syntax for first order inference
David McAllester and Robert Givan · 1993
Earlier work this paper cites.
Production matching for large learning systems
Robert B Doorenbos · 1995
Earlier work this paper cites.
Solution of the robbins problem
William McCune · 1997
Earlier work this paper cites.
Handbook of automated reasoning , volume 1
Alan JA Robinson and Andrei Voronkov · 2001
Earlier work this paper cites.
Unification with sequence variables and flexible arity symbols and its extension with pattern-terms
Temur Kutsia · 2002
Earlier work this paper cites.
Artificial intelligence: a modern approach
Stuart Russell and Peter Norvig · 2002
Earlier work this paper cites.
A theorem proving approach to analysis of secure information flow
Ádám Darvas, Reiner Hähnle, and David Sands · 2005
Earlier work this paper cites.
Natural logic for textual inference
Bill MacCartney and Christopher D Manning · 2007
Earlier work this paper cites.
Z3: An efficient smt solver
Leonardo De Moura and Nikolaj Bjørner · 2008
Earlier work this paper cites.
Towards a formal distributional semantics: Simulating logical calculi with tensors
Original
Edward Grefenstette · 2013
Earlier work this paper cites.
First-order theorem proving and vampire
Laura Kovács and Andrei Voronkov · 2013
Earlier work this paper cites.
Recent progress on monotonicity
Thomas F Icard III and Lawrence S Moss · 2014
Earlier work this paper cites.
Anti-unification for unranked terms and hedges
Temur Kutsia, Jordi Levy, and Mateu Villaret · 2014
Earlier work this paper cites.
Higher-order logical inference with compositional semantics
Koji Mineshima, Pascual Martínez-Gómez, Yusuke Miyao, and Daisuke Bekki · 2015
Earlier work this paper cites.
Deepmath - deep sequence models for premise selection
Alex A Alemi, François Chollet, Niklas Een, Geoffrey Irving, Christian Szegedy, and Josef Urban · 2016
Earlier work this paper cites.
Reasoning in vector space: An exploratory study of question answering
Moontae Lee, Xiaodong He, Wen-tau Yih, Jianfeng Gao, Li Deng, and Paul Smolensky · 2016
Earlier work this paper cites.

Similar

The enigma of reason
Hugo Mercier and Dan Sperber · 2017
Cited alongside, same era.
End-to-end differentiable proving
Tim Rocktäschel and Sebastian Riedel · 2017
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin · 2017
Cited alongside, same era.
Differentiable learning of logical rules for knowledge base reasoning
Fan Yang, Zhilin Yang, and William W Cohen · 2017
Cited alongside, same era.
The conll–sigmorphon 2018 shared task: Universal morphological reinflection
Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Arya D McCarthy, Katharina Kann, Sabrina J Mielke, Garrett Nicolai, Miikka Silfverberg, et al · 2018
Cited alongside, same era.
Neural logic machines
Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, and Denny Zhou · 2018
Cited alongside, same era.
Learning explanatory rules from noisy data
Richard Evans and Edward Grefenstette · 2018
Cited alongside, same era.
Tensorlog: Deep learning meets probabilistic databases
William W Cohen Fan Yang Kathryn and Rivard Mazaitis · 2018
Cited alongside, same era.
Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks
Brenden Lake and Marco Baroni · 2018
Cited alongside, same era.
The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision
Jiayuan Mao, Chuang Gan, Pushmeet Kohli, Joshua B Tenenbaum, and Jiajun Wu · 2018
Cited alongside, same era.
Learning to reason in large theories without imitation
Original
Kshitij Bansal, Christian Szegedy, Markus N Rabe, Sarah M Loos, and Viktor Toman · 2019
Cited alongside, same era.
Bridging machine learning and logical reasoning by abductive learning
Wang-Zhou Dai, Qiuling Xu, Yang Yu, and Zhi-Hua Zhou · 2019
Cited alongside, same era.

Then

Anti-unification in constraint logic programming
Gonzague Yernaux and Wim Vanhoof · 2019
Later among the works it cites.
Compositional generalization via neural-symbolic stack machines
Xinyun Chen, Chen Liang, Adams Wei Yu, Dawn Song, and Denny Zhou · 2020
Later among the works it cites.
Learning invariants through soft unification
Nuri Cingillioglu and Alessandra Russo · 2020
Later among the works it cites.
Transformers as soft reasoners over language
Peter Clark, Oyvind Tafjord, and Kyle Richardson · 2020
Later among the works it cites.
Inductive logic programming at 30: a new introduction
Original
Andrew Cropper and Sebastijan Dumančić · 2020
Later among the works it cites.
Measuring systematic generalization in neural proof generation with transformers
Nicolas Gontier, Koustuv Sinha, Siva Reddy, and Chris Pal · 2020
Later among the works it cites.
Closed loop neural-symbolic learning via integrating neural perception, grammar parsing, and symbolic reasoning
Qing Li, Siyuan Huang, Yining Hong, Yixin Chen, Ying Nian Wu, and Song-Chun Zhu · 2020
Later among the works it cites.
Compositional generalization by learning analytical expressions
Qian Liu, Shengnan An, Jian-Guang Lou, Bei Chen, Zeqi Lin, Yan Gao, Bin Zhou, Nanning Zheng, and Dongmei Zhang · 2020
Later among the works it cites.
Insights for ai from the human mind
Gary Marcus and Ernest Davis · 2020
Later among the works it cites.
Differentiable reasoning on large knowledge bases and natural language
Pasquale Minervini, Matko Bošnjak, Tim Rocktäschel, Sebastian Riedel, and Edward Grefenstette · 2020
Later among the works it cites.
Learning compositional rules via neural program synthesis
Maxwell I Nye, Armando Solar-Lezama, Joshua B Tenenbaum, and Brenden M Lake · 2020
Later among the works it cites.
Generative language modeling for automated theorem proving
Original
Stanislas Polu and Ilya Sutskever · 2020
Later among the works it cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu · 2020
Later among the works it cites.
Prover: Proof generation for interpretable reasoning over rules
Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava, and Mohit Bansal · 2020
Later among the works it cites.
Proofwriter: Generating implications, proofs, and abductive statements over natural language
Original
Oyvind Tafjord, Bhavana Dalvi Mishra, and Peter Clark · 2020
Later among the works it cites.
olmpics-on what language model pre-training captures
Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant · 2020
Later among the works it cites.
Learning to recombine and resample data for compositional generalization
Ekin Akyürek, Afra Feyza Akyürek, and Jacob Andreas · 2021
Closest in time.
Flexible generation of natural language deductions
Kaj Bostrom, Xinyu Zhao, Swarat Chaudhuri, and Greg Durrett · 2021
Closest in time.
Rnnlogic: Learning logic rules for reasoning on knowledge graphs
Meng Qu, Junkun Chen, Louis-Pascal Xhonneux, Yoshua Bengio, and Jian Tang · 2021
Closest in time.
A generative symbolic model for more general natural language understanding and reasoning
Original
Abulhair Saparov and Tom M Mitchell · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…