2020

Efficient Second-Order TreeCRF for Neural Dependency Parsing

Zhang, Yu, Li, Zhenghua, Zhang, Min

Understand

In the deep learning (DL) era, parsing models are extremely simplified with little hurt on performance, thanks to the remarkable capability of multi-layer BiLSTMs in context representation.

As the most popular graph-based dependency parser due to its high efficiency and performance, the biaffine parser directly scores single dependencies under the arc-factorization assumption, and adopts a very simple local token-wise cross-entropy training loss.
This paper for the first time presents a second-order TreeCRF extension to the biaffine parser.
For a long time, the complexity and inefficiency of the inside-outside algorithm hinder the popularity of TreeCRF.

Built on

Finding optimum branchings
Robert Endre Tarjan. 1977 · 1977
Earlier work this paper cites.
Inside-outside reestimation from partially bracketed corpora
Fernando Pereira and Yves Schabes. 1992 · 1992
Earlier work this paper cites.
Supervised grammar induction using training data with limited constituent information
Rebecca Hwa. 1999 · 1999
Earlier work this paper cites.
Bilexical grammars and their cubic-time parsing algorithms
Jason Eisner. 2000 · 2000
Earlier work this paper cites.
Pseudo-projective dependency parsing
Joakim Nivre and Jens Nilsson. 2005 · 2005
Earlier work this paper cites.
Online learning of approximate dependency parsing algorithms
Ryan McDonald and Fernando Pereira. 2006 · 2006
Earlier work this paper cites.
Experiments with a higher-order projective dependency parser
Xavier Carreras. 2007 · 2007
Earlier work this paper cites.
Probabilistic models of nonprojective dependency trees
David A. Smith and Noah A. Smith. 2007 · 2007
Earlier work this paper cites.
Efficient, feature-based, conditional random field parsing
Jenny Rose Finkel, Alex Kleeman, and Christopher D. Manning. 2008 · 2008
Earlier work this paper cites.
The CoNLL-2009 shared task: Syntactic and semantic dependencies in multiple languages
Jan Hajič, Massimiliano Ciaramita, Richard Johansson, Daisuke Kawahara, Maria Antònia Martí, Lluís Màrquez, Adam Meyers, Joakim Nivre, Sebastian Padó, Jan Štěpánek, Pavel Straňák, Mihai Surdeanu, Nianwen Xue, and Yi Zhang. 2009 · 2009
Earlier work this paper cites.
Efficient third-order dependency parsers
Terry Koo and Michael Collins. 2010 · 2010
Earlier work this paper cites.
Fourth-order dependency parsing
Xuezhe Ma and Hai Zhao. 2012 · 2012
Earlier work this paper cites.
A fast and accurate dependency parser using neural networks
Danqi Chen and Christopher Manning. 2014 · 2014
Earlier work this paper cites.

Similar

The inside-outside recursive neural network model for dependency parsing
Phong Le and Willem Zuidema. 2014 · 2014
Cited alongside, same era.
Squibs: Constrained arc-eager dependency parsing
Joakim Nivre, Yoav Goldberg, and Ryan McDonald. 2014 · 2014
Cited alongside, same era.
An effective neural network model for graph-based dependency parsing
Wenzhe Pei, Tao Ge, and Baobao Chang. 2015 · 2015
Cited alongside, same era.
Inside-outside and forward-backward algorithms are just backprop (tutorial paper)
Jason Eisner. 2016 · 2016
Cited alongside, same era.
Simple and accurate dependency parsing using bidirectional LSTM feature representations
Eliyahu Kiperwasser and Yoav Goldberg. 2016 · 2016
Cited alongside, same era.
Neural architectures for named entity recognition
Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016 · 2016
Cited alongside, same era.
Active learning for dependency parsing with partial annotation
Zhenghua Li, Min Zhang, Yue Zhang, Zhanyi Liu, Wenliang Chen, Hua Wu, and Haifeng Wang. 2016 · 2016
Cited alongside, same era.
Universal dependencies v1: A multilingual treebank collection
Joakim Nivre, Marie-Catherine de Marneffe, Filip Ginter, Yoav Goldberg, Jan Hajic, Christopher D. Manning, Ryan McDonald, Slav Petrov, Sampo Pyysalo, Natalia Silveira, Reut Tsarfaty, and Daniel Zeman. 2016 · 2016
Cited alongside, same era.
Graph-based dependency parsing with bidirectional LSTM
Wenhui Wang and Baobao Chang. 2016 · 2016
Cited alongside, same era.
CRF autoencoder for unsupervised dependency parsing
Jiong Cai, Yong Jiang, and Kewei Tu. 2017 · 2017
Cited alongside, same era.
Deep biaffine attention for neural dependency parsing
Timothy Dozat and Christopher D. Manning. 2017 · 2017
Cited alongside, same era.
Stanford’s graph-based neural dependency parser at the CoNLL 2017 shared task
Timothy Dozat, Peng Qi, and Christopher D. Manning. 2017 · 2017
Cited alongside, same era.

Then

What’s going on in neural constituency parsers? an analysis
David Gaddy, Mitchell Stern, and Dan Klein. 2018 · 2018
Later among the works it cites.
Supervised treebank conversion: Data and approaches
Xinzhou Jiang, Zhenghua Li, Bo Zhang, Min Zhang, Sheng Li, and Luo Si. 2018 · 2018
Later among the works it cites.
Deep contextualized word representations
Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018 · 2018
Later among the works it cites.
Graph attention networks
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018 · 2018
Later among the works it cites.
CoNLL 2018 shared task: Multilingual parsing from raw text to universal dependencies
Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018 · 2018
Later among the works it cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019 · 2019
Later among the works it cites.
Unsupervised latent tree induction with deep inside-outside recursive auto-encoders
Andrew Drozdov, Patrick Verga, Mohit Yadav, Mohit Iyyer, and Andrew McCallum. 2019 · 2019
Later among the works it cites.
The (non-)utility of structural features in BiLSTM-based dependency parsers
Agnieszka Falenska and Jonas Kuhn. 2019 · 2019
Later among the works it cites.
Graph-based dependency parsing with graph neural networks
Tao Ji, Yuanbin Wu, and Man Lan. 2019 · 2019
Later among the works it cites.
Self-attentive biaffine dependency parsing
Ying Li, Zhenghua Li, Min Zhang, Rui Wang, Sheng Li, and Luo Si. 2019 · 2019
Later among the works it cites.
Overview of the nlpcc 2019 shared task: cross-domain dependency parsing
Xue Peng, Zhenghua Li, Min Zhang, Rui Wang, Yue Zhang, and Luo Si. 2019 · 2019
Later among the works it cites.
Second-order semantic dependency parsing with end-to-end neural networks
Xinyu Wang, Jingxian Huang, and Kewei Tu. 2019 · 2019
Later among the works it cites.
An empirical investigation of structured output modeling for graph-based neural dependency parsing
Zhisong Zhang, Xuezhe Ma, and Eduard Hovy. 2019 · 2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…