Fetching the paper…
Reading the bibliography…
We present neural machine translation (NMT) models inspired by echo state network (ESN), named Echo State NMT (ESNMT), in which the encoder and decoder layer weights are randomly generated then fixed throughout training.
Richness of deep echo state network dynamics
Claudio Gallicchio and Alessio Micheli. 2019b · 1903
Earlier work this paper cites.
Reservoir topology in deep echo state networks
Claudio Gallicchio and Alessio Micheli. 2019a · 1909
Earlier work this paper cites.
Joseph Enguehard, Dan Busbridge, Vitalii Zhelezniak, and Nils Hammerla. 2019 · 1910
Earlier work this paper cites.
Mind as motion
Jeffrey L. Elman. 1995 · 1995
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997
Earlier work this paper cites.
The “echo state” approach to analysing and training recurrent neural networks-with an erratum note’
Herbert Jaeger. 2001 · 2001
Earlier work this paper cites.
Real-time computing without stable states: A new framework for neural computation based on perturbations
Wolfgang Maass, Thomas Natschläger, and Henry Markram. 2002 · 2002
Earlier work this paper cites.
Extreme learning machine: Theory and applications
Guang-Bin Huang, Qin-Yu Zhu, and Chee Kheong Siew. 2006 · 2006
Earlier work this paper cites.
What makes a dynamical system computationally powerful? , 1 edition, pages 127–154. MIT Press
Robert Albin Legenstein and Wolfgang Maass. 2007 · 2007
Earlier work this paper cites.
Learning grammatical structure with echo state networks
Matthew H. Tong, Adam D. Bickett, Eric M. Christiansen, and Garrison W. Cottrell. 2007 · 2007
Cited alongside, same era.
An experimental unification of reservoir computing methods
David Verstraeten, Benjamin Schrauwen, Michiel D’Haene, and Dirk Stroobandt. 2007 · 2007
Cited alongside, same era.
A neurodynamical model for working memory
Razvan Pascanu and Herbert Jaeger. 2011 · 2011
Cited alongside, same era.
Information processing capacity of dynamical systems
Joni Dambre, David Verstraeten, Benjamin Schrauwen, and Serge Massar. 2012 · 2012
Cited alongside, same era.
On-line processing of grammatical structure using reservoir computing
Xavier Hinaut and Peter F. Dominey. 2012 · 2012
Cited alongside, same era.
Re-visiting the echo state property
Izzet B. Yildiz, Herbert Jaeger, and Stefan J. Kiebel. 2012 · 2012
Cited alongside, same era.
Neural machine translation of rare words with subword units
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016 · 2016
Later among the works it cites.
Deep reservoir computing: A critical experimental analysis
Claudio Gallicchio, Alessio Micheli, and Luca Pedrelli. 2017 · 2017
Later among the works it cites.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Later among the works it cites.
The best of both worlds: Combining recent advances in neural machine translation
Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. 2018 · 2018
Later among the works it cites.
Language modeling teaches you more than translation does: Lessons learned through auxiliary syntactic task analysis
Kelly Zhang and Samuel Bowman. 2018 · 2018
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
An echo state network with working memories for probabilistic language modeling
Yukinori Homma and Masafumi Hagiwara. 2013 · 2013
Cited alongside, same era.
On the difficulty of training recurrent neural networks
Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013 · 2013
Cited alongside, same era.
Pre-wiring and pre-training: What does a neural network need to learn truly general identity rules?
Raquel G. Alhama and Willem H. Zuidema. 2016 · 2016
Cited alongside, same era.
Deep echo state network (deepesn): A brief survey
Claudio Gallicchio and Alessio Micheli. 2017a
Cited in the paper.
Echo state property of deep reservoir computing networks
Claudio Gallicchio and Alessio Micheli. 2017b
Cited in the paper.
Echo state networks for named entity recognition
Rajkumar Ramamurthy, Robin Stenzel, Rafet Sifa, Anna Ladi, and Christian Bauckhage. 2019 · 2019
Later among the works it cites.
What do you learn from context? probing for sentence structure in contextualized word representations
Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R Thomas McCoy, Najoung Kim, Benjamin Van Durme, Sam Bowman, Dipanjan Das, and Ellie Pavlick. 2019 · 2019
Later among the works it cites.
No training required: Exploring random encoders for sentence classification
John Wieting and Douwe Kiela. 2019 · 2019
Later among the works it cites.