Fetching the paper…
Reading the bibliography…
Unsupervised representation learning has recently received lots of interest due to its powerful generalizability through effectively leveraging large-scale unlabeled data.
Classification with hybri generative/discriminative models
R. Raina, Y. Shen, A. Mccallum, and A. Y. Ng · 2004
Earlier work this paper cites.
Principled hybrids of generative and discriminative models
J. A. Lasserre, C. M. Bishop, and T. P. Minka · 2006
Earlier work this paper cites.
Visualizing data using t-SNE
L. van der Maaten and G. Hinton · 2008
Earlier work this paper cites.
ImageNet: A Large-Scale Hierarchical Image Database
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei · 2009
Earlier work this paper cites.
On the generative-discriminative tradeoff approach: Interpretation, asymptotic, efficiency and classification performance
J.-H. Xue and D. M. Titterinton · 2010
Earlier work this paper cites.
Generative adversarial nets
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio · 2014
Earlier work this paper cites.
Layer normalization
J. L. Ba, J. R. Kiros, and G. E. Hinton · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
K. He, X. Zhang, S. Ren, and J. Sun · 2016
Earlier work this paper cites.
Pixel recurrent neural networks
A. V. Oord, N. Kalchbrenner, and K. Kavukcuoglu · 2016
Earlier work this paper cites.
S. Zagoruyko and N. Komodakis · 2016
Earlier work this paper cites.
On calibration of modern neural networks
C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger · 2017
Earlier work this paper cites.
Gans trained by a two time-scale update rule converge to a local nash equilibrium
M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter · 2017
Earlier work this paper cites.
Categorical reparameterization with gumbel-softmax
E. Jang, S. Gu, and B. Poole · 2017
Earlier work this paper cites.
Sgdr: Stochastic gradient descent with warm restarts
I. Loshchilov and F. Hutter · 2017
Earlier work this paper cites.
Attention is all you need
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin · 2017
Earlier work this paper cites.
Bert: Pre-training of deep bidirectional transformers for language understanding
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova · 2018
Earlier work this paper cites.
A simple unified framework for detecting out-of-distribution samples and adversarial attacks
K. Lee, K. Lee, H. Lee, and J. Shin · 2018
Earlier work this paper cites.
Wasserstein introspective neural networks
K. Lee, W. Xu, F. Fan, and Z. Tu · 2018
Earlier work this paper cites.
Improving language understanding by generative pre-training
A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever · 2018
Cited alongside, same era.
Representation learning with contrastive predictive coding
A. van den Oord, Y. Li, and O. Vinyals · 2018
Cited alongside, same era.
Residual flows for invertible generative modeling
R. T. Q. Chen, J. Behrmann, D. Duvenaud, and J.-H. Jacobsen · 2019
Cited alongside, same era.
Generating long sequences with sparse transformers
R. Child, S. Gray, A. Radford, and I. Sutskever · 2019
Cited alongside, same era.
Large scale adversarial representation learning
J. Donahue and K. Simonyan · 2019
Cited alongside, same era.
Implicit generation and modeling with energy-based models
Y. Du and I. Mordatch · 2019
Generative pretraining from pixels
M. Chen, A. Radford, R. Child, J. Wu, H. Jun, P. Dhariwal, D. Luan, and I. Sutskever · 2020
Later among the works it cites.
A simple framework for contrastive learning of visual representations
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton · 2020
Later among the works it cites.
Improved baselines with momentum contrastive learning
X. Chen, H. Fan, R. Girshick, and K. He · 2020
Later among the works it cites.
Electra: Pre-training text encoders as discriminators rather than generators
K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning · 2020
Later among the works it cites.
How well do self-supervised models transfer?
L. Ericsson, H. Gouk, and T. M. Hospedales · 2020
Later among the works it cites.
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Learning deep representations by mutual information estimation and maximization
R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, and Y. Bengio · 2019
Cited alongside, same era.
Axial attention in multidimensional transformers
J. Ho, N. Kalchbrenner, D. Weissenborn, and T. Salimans · 2019
Cited alongside, same era.
Spanbert: Improving pre-training by representing and predicting spans
M. Joshi, D. Chen, Y. Liu, D. S. Weld, L. Zettlemoyer, and O. Levy · 2019
Cited alongside, same era.
A mutual information maximization perspective of language representation learning
L. Kong, C. de Masson d’Autume, W. Ling, L. Yu, Z. Dai, and D. Yogatama · 2019
Cited alongside, same era.
Albert: A lite bert for self-supervised learning of language representations
Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut · 2019
Cited alongside, same era.
M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer · 2019
Cited alongside, same era.
Your classifier is secretly an energy based model and you should treat it like one
W. Grathwohl, K.-C. Wang, J.-H. Jacobsen, D. Duvenaud, M. Norouzi, and K. Swersky · 2020
Later among the works it cites.
Bootstrap your own latent: A new approach to self-supervised learning
J.-B. Grill, F. Strub, F. Altché, C. Tallec, P. H. Richemond, E. Buchatskaya, C. Doersch, B. A. Pires, Z. D. Guo, M. G. Azar, B. Piot, K. Kavukcuoglu, R. Munos, and M. Valko · 2020
Later among the works it cites.
Momentum contrast for unsupervised visual representation learning
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick · 2020
Later among the works it cites.
Distribution augmentation for generative modeling
H. Jun, R. Child, M. Chen, J. Schulman, A. Ramesh, A. Radford, and I. Sutskever · 2020
Later among the works it cites.
i-mix: A strategy for regularizing contrastive representation learning
K. Lee, Y. Zhu, K. Sohn, C.-L. Li, J. Shin, and H. Lee · 2020
Later among the works it cites.
Hybrid discriminative-generative training via contrastive learning
H. Liu and P. Abbeel · 2020
Later among the works it cites.
Self-supervised learning: Generative or contrastive
X. Liu, F. Zhang, Z. Hou, Z. Wang, L. Mian, J. Zhang, and J. Tang · 2020
Later among the works it cites.
Generative classifiers as a basis for trustworthy image classification
R. Mackowiak, L. Ardizzone, U. Köthe, and C. Rother · 2020
Later among the works it cites.
Self-supervised learning of pretext-invariant representations
I. Misra and L. van der Maaten · 2020
Later among the works it cites.
What makes for good views for contrastive learning
Y. Tian, C. Sun, B. Poole, D. Krishnan, C. Schmid, and P. Isola · 2020
Later among the works it cites.
Contrastive training for improved out-of-distribution detection
J. Winkens, R. Bunel, A. G. Roy, R. Stanforth, V. Natarajan, J. R. Ledsam, P. MacWilliams, P. Kohli, A. Karthikesalingam, S. Kohl, T. Cemgil, S. M. A. Eslami, and O. Ronneberger · 2020
Later among the works it cites.
Viewmaker networks: Learning views for unsupervised representation learning
A. Tamkin, M. Wu, and N. Goodman · 2021
Closest in time.