Built on

W. Karush, “Minima of functions of several variables with inequalities as side constraints,” M. Sc. Dissertation. Dept. of Mathematics, Univ. of Chicago
1939
Earlier work this paper cites.
A. K. Wong and M. You, “Entropy and distance of random graphs with application to structural pattern recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence
1985
Earlier work this paper cites.
J. Lin, “Divergence measures based on the shannon entropy,” IEEE Transactions on Information theory
1991
Earlier work this paper cites.
K. Lang, “Newsweeder: Learning to filter netnews,” in Machine Learning Proceedings 1995
1995
Earlier work this paper cites.
A. Blum and T. Mitchell, “Combining labeled and unlabeled data with co-training,” in Proceedings of the eleventh annual conference on Computational learning theory
1998
Earlier work this paper cites.
D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” the Journal of machine Learning research
2003
Earlier work this paper cites.
Y. Lu, Q. Mei, and C. Zhai, “Investigating task performance of probabilistic topic models: an empirical study of plsa and lda,” Information Retrieval
2011
Earlier work this paper cites.
A. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, “Learning word vectors for sentiment analysis,” in Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies
2011
Earlier work this paper cites.
C. Xu, D. Tao, and C. Xu, “A survey on multi-view learning,” arXiv preprint arXiv:1304.5634
Original
2013
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114
Original
2013
Earlier work this paper cites.
D. J. Rezende, S. Mohamed, and D. Wierstra, “Stochastic backpropagation and approximate inference in deep generative models,” in International conference on machine learning
2014
Earlier work this paper cites.
H. W. Kuhn and A. W. Tucker, “Nonlinear programming,” in Traces and emergence of nonlinear programming
2014
Earlier work this paper cites.
Y. Miao, L. Yu, and P. Blunsom, “Neural variational inference for text processing,” in International conference on machine learning
2016
Earlier work this paper cites.
S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer sentinel mixture models,” arXiv preprint arXiv:1609.07843
Original
2016
Earlier work this paper cites.
Y. Miao, E. Grefenstette, and P. Blunsom, “Discovering discrete latent topics with neural variational inference,” in International Conference on Machine Learning
2017
Earlier work this paper cites.
A. Srivastava and C. Sutton, “Autoencoding variational inference for topic models,” arXiv preprint arXiv:1703.01488
Original
2017
Earlier work this paper cites.
D. Card, C. Tan, and N. A. Smith, “Neural models for documents with metadata,” arXiv preprint arXiv:1705.09296
Original
2017
Earlier work this paper cites.
S. Subramani, V. Sridhar, and K. Shetty, “A novel approach of neural topic modelling for document clustering,” in 2018 IEEE Symposium Series on Computational Intelligence (SSCI)
2018
Earlier work this paper cites.
R. Ding, R. Nallapati, and B. Xiang, “Coherence-aware neural topic modeling,” arXiv preprint arXiv:1809.02687
Original
2018
Earlier work this paper cites.

Similar

R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, and Y. Bengio, “Learning deep representations by mutual information estimation and maximization,” arXiv preprint arXiv:1808.06670
Original
2018
Cited alongside, same era.
T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training: a regularization method for supervised and semi-supervised learning,” IEEE transactions on pattern analysis and machine intelligence
2018
Cited alongside, same era.
L. Logeswaran and H. Lee, “An efficient framework for learning sentence representations,” arXiv preprint arXiv:1803.02893
Original
2018
Cited alongside, same era.
A. v. d. Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748
Original
2018
Cited alongside, same era.

Then

J. Robinson, C.-Y. Chuang, S. Sra, and S. Jegelka, “Contrastive learning with hard negative samples,” arXiv preprint arXiv:2010.04592
Original
2020
Later among the works it cites.
T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. Hinton, “Big self-supervised models are strong semi-supervised learners,” arXiv preprint arXiv:2006.10029
Original
2020
Later among the works it cites.
A. Hoyle, P. Goel, and P. Resnik, “Improving neural topic models using knowledge distillation,” arXiv preprint arXiv:2010.02377
Original
2020
Later among the works it cites.
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,” arXiv preprint arXiv:2004.11362
Original
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2021

Contrastive Learning for Neural Topic Model

Nguyen, Thong, Luu, Anh Tuan

Understand

Recent empirical studies show that adversarial topic models (ATM) can successfully capture semantic patterns of the document by differentiating a document with another dissimilar sample.

However, utilizing that discriminative-generative architecture has two important drawbacks: (1) the architecture does not relate similar documents, which has the same document-word distribution of salient words; (2) it restricts the ability to integrate external information, such as sentiments of the document, which has been shown to benefit the training of neural topic model.
To address those issues, we revisit the adversarial topic architecture in the viewpoint of mathematical analysis, propose a novel approach to re-formulate discriminative goal as an optimization problem, and design a novel sampling method which facilitates the integration of external variables.
The reformulation encourages the model to incorporate the relations among similar samples and enforces the constraint on the similarity among dissimilar ones; while the sampling method, which is based on the internal input and reconstructed output, helps inform the model of salient words contributing to the main topic.

Built on

W. Karush, “Minima of functions of several variables with inequalities as side constraints,” M. Sc. Dissertation. Dept. of Mathematics, Univ. of Chicago
1939
Earlier work this paper cites.
A. K. Wong and M. You, “Entropy and distance of random graphs with application to structural pattern recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence
1985
Earlier work this paper cites.
J. Lin, “Divergence measures based on the shannon entropy,” IEEE Transactions on Information theory
1991
Earlier work this paper cites.
K. Lang, “Newsweeder: Learning to filter netnews,” in Machine Learning Proceedings 1995
1995
Earlier work this paper cites.
A. Blum and T. Mitchell, “Combining labeled and unlabeled data with co-training,” in Proceedings of the eleventh annual conference on Computational learning theory
1998
Earlier work this paper cites.
D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” the Journal of machine Learning research
2003
Earlier work this paper cites.
Y. Lu, Q. Mei, and C. Zhai, “Investigating task performance of probabilistic topic models: an empirical study of plsa and lda,” Information Retrieval
2011
Earlier work this paper cites.
A. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, “Learning word vectors for sentiment analysis,” in Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies
2011
Earlier work this paper cites.
C. Xu, D. Tao, and C. Xu, “A survey on multi-view learning,” arXiv preprint arXiv:1304.5634
Original
2013
Earlier work this paper cites.
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114
Original
2013
Earlier work this paper cites.
D. J. Rezende, S. Mohamed, and D. Wierstra, “Stochastic backpropagation and approximate inference in deep generative models,” in International conference on machine learning
2014
Earlier work this paper cites.
H. W. Kuhn and A. W. Tucker, “Nonlinear programming,” in Traces and emergence of nonlinear programming
2014
Earlier work this paper cites.
Y. Miao, L. Yu, and P. Blunsom, “Neural variational inference for text processing,” in International conference on machine learning
2016
Earlier work this paper cites.
S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer sentinel mixture models,” arXiv preprint arXiv:1609.07843
Original
2016
Earlier work this paper cites.
Y. Miao, E. Grefenstette, and P. Blunsom, “Discovering discrete latent topics with neural variational inference,” in International Conference on Machine Learning
2017
Earlier work this paper cites.
A. Srivastava and C. Sutton, “Autoencoding variational inference for topic models,” arXiv preprint arXiv:1703.01488
Original
2017
Earlier work this paper cites.
D. Card, C. Tan, and N. A. Smith, “Neural models for documents with metadata,” arXiv preprint arXiv:1705.09296
Original
2017
Earlier work this paper cites.
S. Subramani, V. Sridhar, and K. Shetty, “A novel approach of neural topic modelling for document clustering,” in 2018 IEEE Symposium Series on Computational Intelligence (SSCI)
2018
Earlier work this paper cites.
R. Ding, R. Nallapati, and B. Xiang, “Coherence-aware neural topic modeling,” arXiv preprint arXiv:1809.02687
Original
2018
Earlier work this paper cites.

Similar

R. D. Hjelm, A. Fedorov, S. Lavoie-Marchildon, K. Grewal, P. Bachman, A. Trischler, and Y. Bengio, “Learning deep representations by mutual information estimation and maximization,” arXiv preprint arXiv:1808.06670
Original
2018
Cited alongside, same era.
T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training: a regularization method for supervised and semi-supervised learning,” IEEE transactions on pattern analysis and machine intelligence
2018
Cited alongside, same era.
L. Logeswaran and H. Lee, “An efficient framework for learning sentence representations,” arXiv preprint arXiv:1803.02893
Original
2018
Cited alongside, same era.
A. v. d. Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748
Original
2018
Cited alongside, same era.

Then

J. Robinson, C.-Y. Chuang, S. Sra, and S. Jegelka, “Contrastive learning with hard negative samples,” arXiv preprint arXiv:2010.04592
Original
2020
Later among the works it cites.
T. Chen, S. Kornblith, K. Swersky, M. Norouzi, and G. Hinton, “Big self-supervised models are strong semi-supervised learners,” arXiv preprint arXiv:2006.10029
Original
2020
Later among the works it cites.
A. Hoyle, P. Goel, and P. Resnik, “Improving neural topic models using knowledge distillation,” arXiv preprint arXiv:2010.02377
Original
2020
Later among the works it cites.
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,” arXiv preprint arXiv:2004.11362
Original
2020
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

S. Jin, A. RoyChowdhury, H. Jiang, A. Singh, A. Prasad, D. Chakraborty, and E. Learned-Miller, “Unsupervised hard example mining from videos for improved object detection,” in Proceedings of the European Conference on Computer Vision (ECCV)

2018

Cited alongside, same era.

R. Wang, D. Zhou, and Y. He, “Open event extraction from online text using a generative adversarial network,” arXiv preprint arXiv:1908.09246