Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations

Neural and conceptual interpretation of PDP models

Paul Smolensky. 1986 · 1986

Earlier work this paper cites.

Finding structure in time

Jeffrey L. Elman. 1990 · 1990

Earlier work this paper cites.

Distributed representations, simple recurrent networks, and grammatical structure

Jeffrey L. Elman. 1991 · 1991

Earlier work this paper cites.

Long short-term memory

Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997

Earlier work this paper cites.

Neural networks for machine learning

Geoffrey Hinton. 2012 · 2012

Earlier work this paper cites.

Estimating or propagating gradients through stochastic neurons for conditional computation

Original

Yoshua Bengio, Nicholas Léonard, and Aaron C. Courville. 2013 · 2013

Earlier work this paper cites.

Linguistic regularities in continuous space word representations

Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013 · 2013

Earlier work this paper cites.

Learning phrase representations using RNN encoder–decoder for statistical machine translation

Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014 · 2014

Earlier work this paper cites.

Linguistic regularities in sparse and explicit word representations

Omer Levy and Yoav Goldberg. 2014 · 2014

Earlier work this paper cites.

A latent variable model approach to PMI-based word embeddings

Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. 2016 · 2016

Earlier work this paper cites.

Layer normalization

Original

Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016 · 2016

Earlier work this paper cites.

Categorical reparametrization with gumbel-softmax

Eric Jang, Shixiang Gu, and Ben Poole. 2017 · 2017

Earlier work this paper cites.

The concrete distribution: A continuous relaxation of discrete random variables

Chris J Maddison, Andriy Mnih, and Yee Whye Teh. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information

Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem H. Zuidema. 2018 · 2018

Cited alongside, same era.

Deep contextualized word representations

Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018 · 2018

Cited alongside, same era.

How do decisions emerge across layers in neural models? interpretation with differentiable masking

Nicola De Cao, Michael Sejr Schlichtkrull, Wilker Aziz, and Ivan Titov. 2020 · 2020

Cited alongside, same era.

Neural natural language inference models partially embed theories of lexical entailment and negation

Atticus Geiger, Kyle Richardson, and Christopher Potts. 2020 · 2020

Cited alongside, same era.

Null it out: Guarding protected attributes by iterative nullspace projection

Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020 · 2020

Cited alongside, same era.

A geometric notion of causal probing

Original

Clément Guerner, Anej Svete, Tianyu Liu, Alexander Warstadt, and Ryan Cotterell. 2023 · 2023

Later among the works it cites.

Emergent linear representations in world models of self-supervised sequence models

Neel Nanda, Andrew Lee, and Martin Wattenberg. 2023 · 2023

Later among the works it cites.

The linear representation hypothesis and the geometry of large language models

Original

Kiho Park, Yo Joong Choe, and Victor Veitch. 2023 · 2023

Later among the works it cites.

Interpretability in the wild: a circuit for indirect object identification in GPT-2 small

Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023 · 2023

Later among the works it cites.

Interpretability at scale: Identifying causal mechanisms in alpaca

Zhengxuan Wu, Atticus Geiger, Thomas Icard, Christopher Potts, and Noah D. Goodman. 2023 · 2023