2020

TAPAS: Weakly Supervised Table Parsing via Pre-training

Herzig, Jonathan, Nowak, Paweł Krzysztof, Müller, Thomas et al.

Understand

Answering natural language questions over tables is usually seen as a semantic parsing task.

To alleviate the collection cost of full logical forms, one popular approach focuses on weak supervision consisting of denotations instead of logical forms.
However, training semantic parsers from weak supervision poses difficulties, and in addition, the generated logical forms are only used as an intermediate step prior to retrieving the denotation.
In this paper, we present TAPAS, an approach to question answering over tables without generating logical forms.

Built on

Cross-lingual language model pretraining
Original
Guillaume Lample and Alexis Conneau. 2019 · 1901
Earlier work this paper cites.
Learning to generalize from sparse and underspecified rewards
Original
R. Agarwal, C. Liang, D. Schuurmans, and M. Norouzi. 2019 · 1902
Earlier work this paper cites.
A comprehensive exploration on wikisql with table-aware word contextualization
Original
Wonseok Hwang, Jinyeung Yim, Seunghyun Park, and Minjoon Seo. 2019 · 1902
Earlier work this paper cites.
Roberta: A robustly optimized BERT pretraining approach
Original
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks
Original
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019 · 1908
Earlier work this paper cites.
Answering conversational questions on structured data without logical forms
Original
Thomas Müller, Francesco Piccinno, Massimo Nicosia, Peter Shaw, and Yasemin Altun. 2019 · 1908
Earlier work this paper cites.
Giving bert a calculator: Finding operations and arguments with reading comprehension
Original
Daniel Andor, Luheng He, Kenton Lee, and Emily Pitler. 2019 · 1909
Earlier work this paper cites.
Tabfact: A large-scale dataset for table-based fact verification
Original
Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2019 · 1909
Earlier work this paper cites.
Do nlp models know numbers? probing numeracy in embeddings
Original
Eric Wallace, Yizhong Wang, Sujian Li, Sameer Singh, and Matt Gardner. 2019 · 1909
Earlier work this paper cites.
Robust estimation of a location parameter
P. J. Huber. 1964 · 1964
Earlier work this paper cites.
Conditional random fields: Probabilistic models for segmenting and labeling data
J. Lafferty, A. McCallum, and F. Pereira. 2001 · 2001
Earlier work this paper cites.
Driving semantic parsing from the world’s response
J. Clarke, D. Goldwasser, M. Chang, and D. Roth. 2010 · 2010
Earlier work this paper cites.
Learning dependency-based compositional semantics
P. Liang, M. I. Jordan, and D. Klein. 2011 · 2011
Earlier work this paper cites.
Weakly supervised learning of semantic parsers for mapping instructions to actions
Y. Artzi and L. Zettlemoyer. 2013 · 2013
Earlier work this paper cites.
Semantic parsing on Freebase from question-answer pairs
J. Berant, A. Chou, R. Frostig, and P. Liang. 2013 · 2013
Earlier work this paper cites.
Semi-supervised sequence learning
A. M. Dai and Q. V. Le. 2015 · 2015
Earlier work this paper cites.

Similar

Neural programmer: Inducing latent programs with gradient descent
Original
Arvind Neelakantan, Quoc V. Le, and Ilya Sutskever. 2015 · 2015
Cited alongside, same era.
Compositional semantic parsing on semi-structured tables
Panupong Pasupat and Percy Liang. 2015 · 2015
Cited alongside, same era.
Building a semantic parser overnight
Y. Wang, J. Berant, and P. Liang. 2015 · 2015
Cited alongside, same era.
Globally normalized transition-based neural networks
Original
D. Andor, C. Alberti, D. Weiss, A. Severyn, A. Presta, K. Ganchev, S. Petrov, and M. Collins. 2016 · 2016
Cited alongside, same era.
Language to logical form with neural attention
L. Dong and M. Lapata. 2016 · 2016
Cited alongside, same era.
Data recombination for neural semantic parsing
R. Jia and P. Liang. 2016 · 2016
Cited alongside, same era.
Neural enquirer: Learning to query tables in natural language
Pengcheng Yin, Zhengdong Lu, Hang Li, and Kao Ben. 2016 · 2016
Cited alongside, same era.
Google Vizier: A Service for Black-Box Optimization
Daniel Golovin, Benjamin Solnik, Subhodeep Moitra, Greg Kochanski, John Elliot Karro, and D. Sculley, editors. 2017 · 2017
Cited alongside, same era.
From language to programs: Bridging reinforcement learning and maximum marginal likelihood
K. Guu, P. Pasupat, E. Z. Liu, and P. Liang. 2017 · 2017
Cited alongside, same era.
Neural semantic parsing over multiple knowledge-bases
J. Herzig and J. Berant. 2017 · 2017
Cited alongside, same era.
Learning a neural semantic parser from user feedback
S. Iyer, I. Konstas, A. Cheung, J. Krishnamurthy, and L. Zettlemoyer. 2017 · 2017
Cited alongside, same era.
Search-based neural structured learning for sequential question answering
M. Iyyer, W. Yih, and M. Chang. 2017 · 2017
Cited alongside, same era.

Then

Adversarial tableqa: Attention supervision for question answering on tables
Minseok Cho, Reinald Kim Amplayo, Seung won Hwang, and Jonghyuck Park. 2018 · 2018
Later among the works it cites.
Neural multi-step reasoning for question answering on semi-structured tables
T. Haug, O. Ganea, and P. Grnarova. 2018 · 2018
Later among the works it cites.
Improving a neural semantic parser by counterfactual learning from human bandit feedback
Carolin Lawrence and Stefan Riezler. 2018 · 2018
Later among the works it cites.
Memory augmented policy optimization for program synthesis with generalization
C. Liang, M. Norouzi, J. Berant, Q. Le, and N. Lao. 2018 · 2018
Later among the works it cites.
Deep contextualized word representations
M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer. 2018 · 2018
Later among the works it cites.
Improving language understanding by generative pre-training
A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever. 2018 · 2018
Later among the works it cites.
Knowledge-aware conversational semantic parsing over web tables
Original
Yibo Sun, Duyu Tang, Nan Duan, Jingjing Xu, Xiaocheng Feng, and Bing Qin. 2018 · 2018
Later among the works it cites.
Iterative search for weakly supervised semantic parsing
Pradeep Dasigi, Matt Gardner, Shikhar Murty, Luke Zettlemoyer, and Eduard Hovy. 2019 · 2019
Later among the works it cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Later among the works it cites.
DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs
D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner. 2019 · 2019
Later among the works it cites.
A discrete hard EM approach for weakly supervised question answering
Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019 · 2019
Later among the works it cites.
Value-based search in execution space for mapping instructions to programs
D. Muhlgay, J. Herzig, and J. Berant. 2019 · 2019
Later among the works it cites.
Probing neural network comprehension of natural language arguments
Timothy Niven and Hung-Yu Kao. 2019 · 2019
Later among the works it cites.
Lxmert: Learning cross-modality encoder representations from transformers
Hao Tan and Mohit Bansal. 2019 · 2019
Later among the works it cites.
Learning semantic parsers from denotations with latent structured alignments and abstract programs
Bailin Wang, Ivan Titov, and Mirella Lapata. 2019 · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…