Fetching the paper…
Reading the bibliography…
There are two major classes of natural language grammar -- the dependency grammar that models one-to-one correspondences between words and the constituency grammar that models the assembly of one or several corresponded words.
Roberta: A robustly optimized bert pretraining approach
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019 · 1907
Earlier work this paper cites.
Exploring the limits of transfer learning with a unified text-to-text transformer
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019 · 1910
Earlier work this paper cites.
Do attention heads in bert track syntactic dependencies?
Phu Mon Htut, Jason Phang, Shikha Bordia, and Samuel R Bowman. 2019 · 1911
Earlier work this paper cites.
Are pre-trained language models aware of phrases? simple but strong baselines for grammar induction
Taeuk Kim, Jihun Choi, Daniel Edmiston, and Sang-goo Lee. 2020 · 2002
Earlier work this paper cites.
A generative constituent-context model for improved grammar induction
Dan Klein and Christopher D Manning. 2002 · 2002
Earlier work this paper cites.
Corpus-based induction of syntactic structure: Models of dependency and constituency
Dan Klein and Christopher D Manning. 2004 · 2004
Earlier work this paper cites.
Transforming a constituency treebank into a dependency treebank
Alexander Gelbukh, Sulema Torres, and Hiram Calvo. 2005 · 2005
Earlier work this paper cites.
A systematic assessment of syntactic generalization in neural language models
Jennifer Hu, Jon Gauthier, Peng Qian, Ethan Wilcox, and Roger P Levy. 2020 · 2005
Earlier work this paper cites.
The unsupervised learning of natural language structure
Dan Klein. 2005 · 2005
Earlier work this paper cites.
Syntactic structure distillation pretraining for bidirectional encoders
Adhiguna Kuncoro, Lingpeng Kong, Daniel Fried, Dani Yogatama, Laura Rimell, Chris Dyer, and Phil Blunsom. 2020 · 2005
Earlier work this paper cites.
Shared logistic normal distributions for soft parameter tying in unsupervised grammar induction
Shay B Cohen and Noah A Smith. 2009 · 2009
Earlier work this paper cites.
Unsupervised search-based structured prediction
Hal Daumé III. 2009 · 2009
Earlier work this paper cites.
Improving unsupervised dependency parsing with richer contexts and smoothing
William P Headden III, Mark Johnson, and David McClosky. 2009 · 2009
Earlier work this paper cites.
Sparsity in dependency grammar induction
Jennifer Gillenwater, Kuzman Ganchev, João Graça, Fernando Pereira, and Ben Taskar. 2010 · 2010
Cited alongside, same era.
Unsupervised structure prediction with non-parallel multilingual guidance
Shay B Cohen, Dipanjan Das, and Noah A Smith. 2011 · 2011
Cited alongside, same era.
Unsupervised dependency parsing without gold part-of-speech tags
Valentin I Spitkovsky, Hiyan Alshawi, Angel Chang, and Dan Jurafsky. 2011 · 2011
Cited alongside, same era.
Statistical language models based on neural networks
Tomáš Mikolov et al. 2012 · 2012
Cited alongside, same era.
Unambiguity regularization for unsupervised learning of probabilistic grammars
Kewei Tu and Vasant Honavar. 2012 · 2012
Cited alongside, same era.
Unsupervised dependency parsing with acoustic cues
John K Pate and Sharon Goldwater. 2013 · 2013
Cited alongside, same era.
Encoding sentences with graph convolutional networks for semantic role labeling
Diego Marcheggiani and Ivan Titov. 2017 · 2017
Later among the works it cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Later among the works it cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Later among the works it cites.
Unsupervised learning of syntactic structure with invertible neural projections
Junxian He, Graham Neubig, and Taylor Berg-Kirkpatrick. 2018 · 2018
Later among the works it cites.
Linguistically-informed self-attention for semantic role labeling
Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. 2018 · 2018
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Breaking out of local optima with count transforms and model recombination: A study in grammar induction
Valentin I Spitkovsky, Daniel Jurafsky, and Hiyan Alshawi. 2013 · 2013
Cited alongside, same era.
Unsupervised dependency parsing: Let’s use supervised parsers
Phong Le and Willem Zuidema. 2015 · 2015
Cited alongside, same era.
Recurrent neural network grammars
Chris Dyer, Adhiguna Kuncoro, Miguel Ballesteros, and Noah A Smith. 2016 · 2016
Cited alongside, same era.
Unsupervised neural dependency parsing
Yong Jiang, Wenjuan Han, Kewei Tu, et al. 2016 · 2016
Cited alongside, same era.
Crf autoencoder for unsupervised dependency parsing
Jiong Cai, Yong Jiang, and Kewei Tu. 2017 · 2017
Cited alongside, same era.
Dependency grammar induction with neural lexicalization and big training data
Wenjuan Han, Yong Jiang, and Kewei Tu. 2017 · 2017
Cited alongside, same era.
Later among the works it cites.
Dependency-based self-attention for transformer nmt
Hiroyuki Deguchi, Akihiro Tamura, and Takashi Ninomiya. 2019 · 2019
Later among the works it cites.
Unsupervised latent tree induction with deep inside-outside recursive auto-encoders
Andrew Drozdov, Patrick Verga, Mohit Yadav, Mohit Iyyer, and Andrew McCallum. 2019 · 2019
Later among the works it cites.
Unsupervised recurrent neural network grammars
Yoon Kim, Alexander M Rush, Lei Yu, Adhiguna Kuncoro, Chris Dyer, and Gábor Melis. 2019b · 2019
Later among the works it cites.
Improving neural language models by segmenting, attending, and predicting the future
Hongyin Luo, Lan Jiang, Yonatan Belinkov, and James Glass. 2019 · 2019
Later among the works it cites.
Dependency-based relative positional encoding for transformer nmt
Yutaro Omote, Akihiro Tamura, and Takashi Ninomiya. 2019 · 2019
Later among the works it cites.
Tree transformer: Integrating tree structures into self-attention
Yaushian Wang, Hung-Yi Lee, and Yun-Nung Chen. 2019 · 2019
Later among the works it cites.
The return of lexical dependencies: Neural lexicalized pcfgs
Hao Zhu, Yonatan Bisk, and Graham Neubig. 2020 · 2020
Closest in time.