Built on

A. Krizhevsky, “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009
2009
Earlier work this paper cites.
Y. LeCun, C. Cortes, and C. Burges, “Mnist handwritten digit database,” ATT Labs. Available: http://yann. lecun. com/exdb/mnist , vol. 2, 2010
2010
Earlier work this paper cites.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” Advances in Neural Information Processing Systems (NIPS) , 2011
2011
Earlier work this paper cites.
A. Van Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” in International conference on machine learning . PMLR, 2016, pp. 1747–1756
2016
Earlier work this paper cites.
H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747 , 2017
Original
2017
Earlier work this paper cites.
G. Cohen, S. Afshar, J. Tapson, and A. V. Schaik, “Emnist: Extending mnist to handwritten letters,” 2017 International Joint Conference on Neural Networks (IJCNN) , 2017
2017
Earlier work this paper cites.
H. Choi, E. Jang, and A. A. Alemi, “Waic, but why? generative ensembles for robust anomaly detection,” arXiv preprint arXiv:1810.01392 , 2018
Original
2018
Earlier work this paper cites.

Similar

D. P. Kingma and P. Dhariwal, “Glow: Generative flow with invertible 1x1 convolutions,” Advances in neural information processing systems , vol. 31, 2018
2018
Cited alongside, same era.
M. Rosca, B. Lakshminarayanan, and S. Mohamed, “Distribution matching in variational inference,” arXiv preprint arXiv:1802.06847 , 2018
Original
2018
Cited alongside, same era.
T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normalization for generative adversarial networks,” in International Conference on Learning Representations , 2018
2018
Cited alongside, same era.
T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha, “Deep learning for classical japanese literature,” arXiv preprint arXiv:1812.01718 , 2018
Original
2018
Cited alongside, same era.

Then

D. Hendrycks, M. Mazeika, and T. Dietterich, “Deep anomaly detection with outlier exposure,” in International Conference on Learning Representations , 2019
2019
Closest in time.
T. Daniel, T. Kurutach, and A. Tamar, “Deep variational semi-supervised novelty detection,” arXiv preprint arXiv:1911.04971 , 2019
Original
2019
Closest in time.
C. Ji and H. Shen, “Stochastic variational inference via upper bound,” arXiv preprint arXiv:1912.00650 , 2019
Original
2019
Closest in time.
A. Csiszárik, B. Benkő, and D. Varga, “Negative sampling in variational autoencoders,” arXiv preprint arXiv:1910.02760 , 2019
Original
2019
Closest in time.
Z. Xiao, Q. Yan, and Y. Amit, “Likelihood regret: An out-of-distribution detection score for variational auto-encoder,” Advances in neural information processing systems , vol. 33, pp. 20 685–20 696, 2020

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2019

Negative Sampling in Variational Autoencoders

Csiszárik, Adrián, Benkő, Beatrix, Varga, Dániel

Understand

Modern deep artificial neural networks have achieved great success in the domain of computer vision and beyond.

However, their application to many real-world tasks is undermined by certain limitations, such as overconfident uncertainty estimates on out-of-distribution data or performance deterioration under data distribution shifts.
Several types of deep learning models used for density estimation through probabilistic generative modeling have been shown to fail to detect out-of-distribution samples by assigning higher likelihoods to anomalous data.
We investigate this failure mode in Variational Autoencoder models, which are also prone to this, and improve upon the out-of-distribution generalization performance of the model by employing an alternative training scheme utilizing negative samples.

Built on

A. Krizhevsky, “Learning multiple layers of features from tiny images,” Citeseer, Tech. Rep., 2009
2009
Earlier work this paper cites.
Y. LeCun, C. Cortes, and C. Burges, “Mnist handwritten digit database,” ATT Labs. Available: http://yann. lecun. com/exdb/mnist , vol. 2, 2010
2010
Earlier work this paper cites.
Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” Advances in Neural Information Processing Systems (NIPS) , 2011
2011
Earlier work this paper cites.
A. Van Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” in International conference on machine learning . PMLR, 2016, pp. 1747–1756
2016
Earlier work this paper cites.
H. Xiao, K. Rasul, and R. Vollgraf, “Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms,” arXiv preprint arXiv:1708.07747 , 2017
Original
2017
Earlier work this paper cites.
G. Cohen, S. Afshar, J. Tapson, and A. V. Schaik, “Emnist: Extending mnist to handwritten letters,” 2017 International Joint Conference on Neural Networks (IJCNN) , 2017
2017
Earlier work this paper cites.
H. Choi, E. Jang, and A. A. Alemi, “Waic, but why? generative ensembles for robust anomaly detection,” arXiv preprint arXiv:1810.01392 , 2018
Original
2018
Earlier work this paper cites.

Similar

D. P. Kingma and P. Dhariwal, “Glow: Generative flow with invertible 1x1 convolutions,” Advances in neural information processing systems , vol. 31, 2018
2018
Cited alongside, same era.
M. Rosca, B. Lakshminarayanan, and S. Mohamed, “Distribution matching in variational inference,” arXiv preprint arXiv:1802.06847 , 2018
Original
2018
Cited alongside, same era.
T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida, “Spectral normalization for generative adversarial networks,” in International Conference on Learning Representations , 2018
2018
Cited alongside, same era.
T. Clanuwat, M. Bober-Irizar, A. Kitamoto, A. Lamb, K. Yamamoto, and D. Ha, “Deep learning for classical japanese literature,” arXiv preprint arXiv:1812.01718 , 2018
Original
2018
Cited alongside, same era.

Then

D. Hendrycks, M. Mazeika, and T. Dietterich, “Deep anomaly detection with outlier exposure,” in International Conference on Learning Representations , 2019
2019
Closest in time.
T. Daniel, T. Kurutach, and A. Tamar, “Deep variational semi-supervised novelty detection,” arXiv preprint arXiv:1911.04971 , 2019
Original
2019
Closest in time.
C. Ji and H. Shen, “Stochastic variational inference via upper bound,” arXiv preprint arXiv:1912.00650 , 2019
Original
2019
Closest in time.
A. Csiszárik, B. Benkő, and D. Varga, “Negative sampling in variational autoencoders,” arXiv preprint arXiv:1910.02760 , 2019
Original
2019
Closest in time.
Z. Xiao, Q. Yan, and Y. Amit, “Likelihood regret: An out-of-distribution detection score for variational auto-encoder,” Advances in neural information processing systems , vol. 33, pp. 20 685–20 696, 2020

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

K. Lee, H. Lee, K. Lee, and J. Shin, “Training confidence-calibrated classifiers for detecting out-of-distribution samples,” in International Conference on Learning Representations , 2018

2018

Cited alongside, same era.

E. Nalisnick, A. Matsukawa, Y. W. Teh, D. Gorur, and B. Lakshminarayanan, “Do deep generative models know what they don’t know?” in International Conference on Learning Representations , 2019

2019

Cited alongside, same era.

D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in International Conference on Learning Representations , 2019

2019

Cited alongside, same era.

2020

Closest in time.

X. Ran, M. Xu, L. Mei, Q. Xu, and Q. Liu, “Detecting out-of-distribution samples via variational auto-encoder with reliable uncertainty estimation,” Neural Networks , vol. 145, pp. 199–208, 2022

2022

Closest in time.