Fetching the paper…

Reading the bibliography…

2014

Variational Recurrent Auto-Encoders

Fabius, Otto, van Amersfoort, Joost R.

Original
PDF
Post on X

Understand

In this paper we propose a model that combines the strengths of RNNs and SGVB: the Variational Recurrent Auto-Encoder (VRAE).

Such a model can be used for efficient, large scale unsupervised learning on time series data, mapping the time series data to a latent vector representation.
The model is generative, such that data can be generated from samples of the latent space.
An important contribution of this work is that the model can make use of unlabeled data in order to facilitate supervised training of RNNs by initialising the weights and network state.

Variational Recurrent Auto-Encoders · Around

Built on

Long short-term memory
Hochreiter, Sepp and Schmidhuber, Jürgen · 1997
Earlier work this paper cites.
Visualizing data using t-sne
Original
Van der Maaten, Laurens and Hinton, Geoffrey · 2008
Earlier work this paper cites.
A novel connectionist system for unconstrained handwriting recognition
Graves, Alex, Liwicki, Marcus, Fernández, Santiago, Bertolami, Roman, Bunke, Horst, and Schmidhuber, Jürgen · 2009
Earlier work this paper cites.
Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription
Boulanger-Lewandowski, Nicolas, Bengio, Yoshua, and Vincent, Pascal · 2012
Earlier work this paper cites.

Similar

Auto-encoding variational bayes
Kingma, Diederik P and Welling, Max · 2013
Cited alongside, same era.
On the difficulty of training recurrent neural networks
Pascanu, Razvan, Mikolov, Tomas, and Bengio, Yoshua · 2013
Cited alongside, same era.
Learning stochastic recurrent networks
Bayer, Justin and Osendorfer, Christian · 2014
Cited alongside, same era.
Learning phrase representations using rnn encoder-decoder for statistical machine translation
Cho, Kyunghyun, van Merrienboer, Bart, Gulcehre, Caglar, Bougares, Fethi, Schwenk, Holger, and Bengio, Yoshua · 2014
Cited alongside, same era.

Then

Adam: A method for stochastic optimization
Original
Kingma, Diederik P and Ba, Jimmy · 2014
Closest in time.
Stochastic backpropagation and approximate inference in deep generative models
Rezende, Danilo Jimenez, Mohamed, Shakir, and Wierstra, Daan · 2014
Closest in time.
An rnn-based music language model for improving automatic music transcription
Sigtia, Siddharth, Benetos, Emmanouil, Cherla, Srikanth, Weyde, Tillman, Garcez, Artur S d’Avila, and Dixon, Simon · 2014
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…