Self-Attention Attribution: Interpreting Information Interactions Inside Transformer

Attention is not Explanation

Original

Jain, S.; and Wallace, B. C. 2019 · 1902

Earlier work this paper cites.

Are Sixteen Heads Really Better than One?

Original

Michel, P.; Levy, O.; and Neubig, G. 2019 · 1905

Earlier work this paper cites.

What Does BERT Look At? An Analysis of BERT’s Attention

Original

Clark, K.; Khandelwal, U.; Levy, O.; and Manning, C. D. 2019 · 1906

Earlier work this paper cites.

Visualizing and Measuring the Geometry of BERT

Original

Coenen, A.; Reif, E.; Yuan, A.; Kim, B.; Pearce, A.; Viégas, F. B.; and Wattenberg, M. 2019 · 1906

Earlier work this paper cites.

From Balustrades to Pierre Vinken: Looking for Syntax in Transformer Self-Attentions

Original

Marecek, D.; and Rosa, R. 2019 · 1906

Earlier work this paper cites.

Inducing Syntactic Trees from BERT Representations

Original

Rosa, R.; and Marecek, D. 2019 · 1906

Earlier work this paper cites.

RoBERTa: A Robustly Optimized BERT Pretraining Approach

Original

Liu, Y.; Ott, M.; Goyal, N.; Du, J.; Joshi, M.; Chen, D.; Levy, O.; Lewis, M.; Zettlemoyer, L.; and Stoyanov, V. 2019 · 1907

Earlier work this paper cites.

Long Short-Term Memory

Hochreiter, S.; and Schmidhuber, J. 1997 · 1997

Earlier work this paper cites.

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

Original

Bao, H.; Dong, L.; Wei, F.; Wang, W.; Yang, N.; Liu, X.; Wang, Y.; Piao, S.; Gao, J.; Zhou, M.; and Hon, H.-W. 2020 · 2002

Earlier work this paper cites.

Automatically constructing a corpus of sentential paraphrases

Dolan, W. B.; and Brockett, C. 2005 · 2005

Earlier work this paper cites.

The second PASCAL recognising textual entailment challenge

Bar-Haim, R.; Dagan, I.; Dolan, B.; Ferro, L.; and Giampiccolo, D. 2006 · 2006

Earlier work this paper cites.

The PASCAL Recognising Textual Entailment Challenge

Dagan, I.; Glickman, O.; and Magnini, B. 2006 · 2006

Earlier work this paper cites.

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training

Original

Chi, Z.; Dong, L.; Wei, F.; Yang, N.; Singhal, S.; Wang, W.; Song, X.; Mao, X.; Huang, H.; and Zhou, M. 2020b · 2007

Earlier work this paper cites.

The Third PASCAL Recognizing Textual Entailment Challenge

Giampiccolo, D.; Magnini, B.; Dagan, I.; and Dolan, B. 2007 · 2007

Cited alongside, same era.

The Fifth PASCAL Recognizing Textual Entailment Challenge

Bentivogli, L.; Dagan, I.; Dang, H. T.; Giampiccolo, D.; and Magnini, B. 2009 · 2009

Cited alongside, same era.

Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank

Socher, R.; Perelygin, A.; Wu, J.; Chuang, J.; Manning, C. D.; Ng, A.; and Potts, C. 2013 · 2013

Cited alongside, same era.

Neural Machine Translation by Jointly Learning to Align and Translate

Bahdanau, D.; Cho, K.; and Bengio, Y. 2015 · 2015

Cited alongside, same era.

Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers

Original

Binder, A.; Montavon, G.; Bach, S.; Müller, K.; and Samek, W. 2016 · 2016

Cited alongside, same era.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019 · 2019

Later among the works it cites.

Unified Language Model Pre-training for Natural Language Understanding and Generation

Dong, L.; Yang, N.; Wang, W.; Wei, F.; Liu, X.; Wang, Y.; Gao, J.; Zhou, M.; and Hon, H.-W. 2019 · 2019

Later among the works it cites.

A Structural Probe for Finding Syntax in Word Representations

Hewitt, J.; and Manning, C. D. 2019 · 2019

Later among the works it cites.

Revealing the Dark Secrets of BERT

Kovaleva, O.; Romanov, A.; Rogers, A.; and Rumshisky, A. 2019 · 2019

Later among the works it cites.

Is Attention Interpretable?

Serrano, S.; and Smith, N. A. 2019 · 2019

Later among the works it cites.

Universal Adversarial Triggers for Attacking and Analyzing NLP

Wallace, E.; Feng, S.; Kandpal, N.; Gardner, M.; and Singh, S. 2019 · 2019