Fetching the paper…
Reading the bibliography…
We incorporate Tensor-Product Representations within the Transformer in order to better support the explicit representation of relation structure.
BERT rediscovers the classical NLP pipeline
Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019 · 1905
Earlier work this paper cites.
Visualizing and measuring the geometry of BERT
Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, and Martin Wattenberg. 2019 · 1906
Earlier work this paper cites.
Open sesame: Getting inside BERT’s linguistic knowledge
Yongjie Lin, Yi Chern Tan, and Robert Frank. 2019 · 1906
Earlier work this paper cites.
A perspective on objects and systematic generalization in model-based rl
Sjoerd van Steenkiste, Klaus Greff, and Jürgen Schmidhuber. 2019 · 1906
Earlier work this paper cites.
Augmenting self-attention with persistent memory
Sainbayar Sukhbaatar, Edouard Grave, Guillaume Lample, Herve Jegou, and Armand Joulin. 2019 · 1907
Earlier work this paper cites.
Stabilizing transformers for reinforcement learning
Emilio Parisotto, H Francis Song, Jack W Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant M Jayakumar, Max Jaderberg, Raphael Lopez Kaufman, Aidan Clark, Seb Noury, et al. 2019 · 1910
Earlier work this paper cites.
Cybernetic Predicting Devices
Alekseĭ Grigorʹevich Ivakhnenko and Valentin Grigorévich Lapa. 1965 · 1965
Earlier work this paper cites.
Polynomial theory of complex systems
Aleksey Grigorievitch Ivakhnenko. 1971 · 1971
Earlier work this paper cites.
The correlation theory of brain function (internal report 81-2)
Christoph von der Malsburg. 1981 · 1981
Earlier work this paper cites.
Distributed representations
Geoffrey E Hinton, James L McClelland, David E Rumelhart, et al. 1984 · 1984
Earlier work this paper cites.
Tensor product variable binding and the representation of symbolic structures in connectionist systems
P. Smolensky. 1990 · 1990
Cited alongside, same era.
On decreasing the ratio between learning complexity and number of time-varying variables in fully recurrent nets
J. Schmidhuber. 1993 · 1993
Cited alongside, same era.
The correlation theory of brain function
Christoph von der Malsburg. 1994 · 1994
Cited alongside, same era.
Learning Task-Dependent Distributed Representations by Backpropagation Through Structure
Christoph Goller and Andreas Küchler. 1995 · 1995
Cited alongside, same era.
Long Short-Term Memory
S. Hochreiter and J. Schmidhuber. 1997 · 1997
Cited alongside, same era.
A new model for learning in graph domains
Marco Gori, Gabriele Monfardini, and Franco Scarselli. 2005 · 2005
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Later among the works it cites.
Systematic generalization: What is required and can it be learned?
Dzmitry Bahdanau, Shikhar Murty, Michael Noukhovitch, Thien Huu Nguyen, Harm de Vries, and Aaron Courville. 2018 · 2018
Later among the works it cites.
Relational inductive biases, deep learning, and graph networks
Peter W Battaglia, Jessica B Hamrick, Victor Bapst, Alvaro Sanchez-Gonzalez, Vinicius Zambaldi, Mateusz Malinowski, Andrea Tacchetti, David Raposo, Adam Santoro, Ryan Faulkner, et al. 2018 · 2018
Later among the works it cites.
BERT: Pre-training of deep bidirectional transformers for language understanding
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018 · 2018
Later among the works it cites.
Relational recurrent neural networks
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Understanding the difficulty of training deep feedforward neural networks
Xavier Glorot and Yoshua Bengio. 2010 · 2010
Cited alongside, same era.
Rupesh Kumar Srivastava, Klaus Greff, and Juergen Schmidhuber. 2015 · 2015
Cited alongside, same era.
Brenden M Lake and Marco Baroni. 2017 · 2017
Cited alongside, same era.
Deep learning of grammatically-interpretable representations through question-answering
Hamid Palangi, Paul Smolensky, Xiaodong He, and Li Deng. 2017 · 2017
Cited alongside, same era.
Adam Santoro, Ryan Faulkner, David Raposo, Jack Rae, Mike Chrzanowski, Theophane Weber, Daan Wierstra, Oriol Vinyals, Razvan Pascanu, and Timothy Lillicrap. 2018 · 2018
Later among the works it cites.
Learning to reason with third order tensor products
Imanol Schlag and Jürgen Schmidhuber. 2018 · 2018
Later among the works it cites.
Learning distributed representations of symbolic structure using binding and unbinding operations
Shuai Tang, Paul Smolensky, and Virginia R de Sa. 2018 · 2018
Later among the works it cites.
A structural probe for finding syntax in word representations
John Hewitt and Christopher D Manning. 2019 · 2019
Closest in time.
Analysing mathematical reasoning abilities of neural models
David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. 2019 · 2019
Closest in time.