From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions

Linguistic knowledge and transferability of contextual representations

Original

Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew Peters, and Noah A. Smith. 2019 · 1903

Earlier work this paper cites.

Dynamic programming parsing for context-free grammars in continuous speech recognition

Hermann Ney. 1991 · 1991

Earlier work this paper cites.

Building a large annotated corpus of english: The penn treebank

Mitchell P Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993 · 1993

Earlier work this paper cites.

Syntactic annotation of a german newspaper corpus

Wojciech Skut, Hans Uszkoreit, and Thorsten Brants. 1999 · 1999

Earlier work this paper cites.

Building a treebank for french

Anne Abeillé, Lionel Clément, and François Toussenel. 2003 · 2003

Earlier work this paper cites.

Fast exact inference with a factored model for natural language parsing

Dan Klein and Christopher D Manning. 2003 · 2003

Earlier work this paper cites.

Europarl: A Parallel Corpus for Statistical Machine Translation

Philipp Koehn. 2005 · 2005

Earlier work this paper cites.

TectoMT: Modular NLP framework

Martin Popel and Zdeněk Žabokrtský. 2010 · 2010

Earlier work this paper cites.

Apertium: a free/open-source platform for rule-based machine translation

Mikel L. Forcada, Mireia Ginestí-Rosell, Jacob Nordfalk, Jim O’Regan, Sergio Ortiz-Rojas, Juan Antonio Pérez-Ortiz, Felipe Sánchez-Martínez, Gema Ramírez-Sánchez, and Francis M. Tyers. 2011 · 2011

Earlier work this paper cites.

The Stanford CoreNLP natural language processing toolkit

Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014 · 2014

Cited alongside, same era.

Fine-grained analysis of sentence embeddings using auxiliary prediction tasks

Original

Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2016 · 2016

Cited alongside, same era.

Assessing the ability of LSTMs to learn syntax-sensitive dependencies

Tal Linzen, Emmanuel Dupoux, and Yoav Goldberg. 2016 · 2016

Cited alongside, same era.

Neural machine translation of rare words with subword units

Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 · 2016

Cited alongside, same era.

Does string-based neural MT learn source syntax?

Xing Shi, Inkit Padhi, and Kevin Knight. 2016 · 2016

Cited alongside, same era.

Attention is all you need

BERT: pre-training of deep bidirectional transformers for language understanding

Original

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018

Later among the works it cites.

Neural monkey: The current state and beyond

Jindřich Helcl, Jindřich Libovický, Tom Kocmi, Tomáš Musil, Ondřej Cífka, Dušan Variš, and Ondřej Bojar. 2018 · 2018

Later among the works it cites.

Extracting syntactic trees from transformer encoder self-attentions

David Mareček and Rudolf Rosa. 2018 · 2018

Later among the works it cites.

Deep contextualized word representations

Original

Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018 · 2018

Later among the works it cites.

An analysis of encoder representations in transformer-based machine translation

Alessandro Raganato and Jörg Tiedemann. 2018 · 2018

Later among the works it cites.