2016

Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision

Liang, Chen, Berant, Jonathan, Le, Quoc et al.

Understand

Harnessing the statistical power of neural networks to perform language understanding and symbolic reasoning is difficult, when it requires executing efficient discrete operations against a large knowledge-base.

In this work, we introduce a Neural Symbolic Machine, which contains (a) a neural "programmer", i.e., a sequence-to-sequence model that maps language utterances to programs and utilizes a key-variable memory to handle compositionality (b) a symbolic "computer", i.e., a Lisp interpreter that performs program execution, and helps find good programs by pruning the search space.
We apply REINFORCE to directly optimize the task reward of this structured prediction problem.
To train with weak supervision and improve the stability of REINFORCE, we augment it with an iterative maximum-likelihood training process.

Built on

Simple statistical gradient-following algorithms for connectionist reinforcement learning
Ronald J. Williams. 1992 · 1992
Earlier work this paper cites.
Learning to parse database queries using inductive logic programming
M. Zelle and R. J. Mooney. 1996 · 1996
Earlier work this paper cites.
Reinforcement Learning: An Introduction
Richard S. Sutton and Andrew G. Barto. 1998 · 1998
Earlier work this paper cites.
Optimal ordered problem solver
Jürgen Schmidhuber. 2004 · 2004
Earlier work this paper cites.
Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars
L. S. Zettlemoyer and M. Collins. 2005 · 2005
Earlier work this paper cites.
Freebase: a collaboratively created graph database for structuring human knowledge
K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor. 2008 · 2008
Earlier work this paper cites.
Search-based structured prediction
H. Daume, J. Langford, and D. Marcu. 2009 · 2009
Earlier work this paper cites.
Random walk inference and learning in a large scale knowledge base
Ni Lao, Tom Mitchell, and William W Cohen. 2011 · 2011
Earlier work this paper cites.
Learning dependency-based compositional semantics
P. Liang, M. I. Jordan, and D. Klein. 2011 · 2011
Earlier work this paper cites.
A reduction of imitation learning and structured prediction to no-regret online learning
S. Ross, G. Gordon, and A. Bagnell. 2011 · 2011
Earlier work this paper cites.
Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups
Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdel-rahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012 · 2012
Earlier work this paper cites.
Learned prioritization for trading off accuracy and speed
J. Jiang, A. Teichert, J. Eisner, and H. Daume. 2012 · 2012
Earlier work this paper cites.
Semantic parsing on freebase from question-answer pairs
Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013 · 2013
Earlier work this paper cites.
Neural machine translation by jointly learning to align and translate
Original
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014 · 2014
Earlier work this paper cites.

Similar

Learning phrase representations using rnn encoder–decoder for statistical machine translation
Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014 · 2014
Cited alongside, same era.
Neural turing machines
Original
Alex Graves, Greg Wayne, and Ivo Danihelka. 2014 · 2014
Cited alongside, same era.
Adam: A method for stochastic optimization
Original
Diederik P. Kingma and Jimmy Ba. 2014 · 2014
Cited alongside, same era.
Glove: Global vectors for word representation
Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014 · 2014
Cited alongside, same era.
Sequence to sequence learning with neural networks
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014 · 2014
Cited alongside, same era.
Memory networks
Original
Jason Weston, Sumit Chopra, and Antoine Bordes. 2014 · 2014
Cited alongside, same era.
Imitation learning of agenda-based semantic parsers
Jonathan Berant and Percy Liang. 2015 · 2015
Cited alongside, same era.
Neural gpus learn algorithms
Original
Łukasz Kaiser and Ilya Sutskever. 2015 · 2015
Cited alongside, same era.
Neural programmer: Inducing latent programs with gradient descent
Original
Arvind Neelakantan, Quoc V. Le, and Ilya Sutskever. 2015 · 2015
Cited alongside, same era.
Compositional semantic parsing on semi-structured tables
Panupong Pasupat and Percy Liang. 2015 · 2015
Cited alongside, same era.
Sequence level training with recurrent neural networks
Original
Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015 · 2015
Cited alongside, same era.
Pointer networks
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. 2015 · 2015
Cited alongside, same era.

Then

Neural enquirer: Learning to query tables
Original
Pengcheng Yin, Zhengdong Lu, Hang Li, and Ben Kao. 2015 · 2015
Later among the works it cites.
Reinforcement learning neural turing machines
Original
Wojciech Zaremba and Ilya Sutskever. 2015 · 2015
Later among the works it cites.
Learning to compose neural networks for question answering
Original
Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. 2016 · 2016
Closest in time.
Language to logical form with neural attention
Li Dong and Mirella Lapata. 2016 · 2016
Closest in time.
Data recombination for neural semantic parsing
Robin Jia and Percy Liang. 2016 · 2016
Closest in time.
Deep reinforcement learning for dialogue generation
Original
Jiwei Li, Will Monroe, Alan Ritter, Michel Galley, Jianfeng Gao, and Dan Jurafsky. 2016 · 2016
Closest in time.
Reward augmented maximum likelihood for neural structured prediction
Mohammad Norouzi, Samy Bengio, zhifeng Chen, Navdeep Jaitly, Mike Schuster, Yonghui Wu, and Dale Schuurmans. 2016 · 2016
Closest in time.
Neural programmer-interpreters
Scott Reed and Nando de Freitas. 2016 · 2016
Closest in time.
Prioritized experience replay
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. 2016 · 2016
Closest in time.
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016 · 2016
Closest in time.
Sequence-to-sequence learning as beam-search optimization
Original
Sam Wiseman and Alexander M. Rush. 2016 · 2016
Closest in time.
Google’s neural machine translation system: Bridging the gap between human and machine translation
Original
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016 · 2016
Closest in time.
The value of semantic parse labeling for knowledge base question answering
Wen-tau Yih, Matthew Richardson, Chris Meek, Ming-Wei Chang, and Jina Suh. 2016 · 2016
Closest in time.
Learning to skim text
Adam Yu, Hongrae Lee, and Quoc Le. 2017 · 2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…