2020

Adversarial Training for Aspect-Based Sentiment Analysis with BERT

Karimi, Akbar, Rossi, Leonardo, Prati, Andrea

Understand

Aspect-Based Sentiment Analysis (ABSA) deals with the extraction of sentiments and their targets.

Collecting labeled data for this task in order to help neural networks generalize better can be laborious and time-consuming.
As an alternative, similar data to the real-world examples can be produced artificially through an adversarial process which is carried out in the embedding space.
Although these examples are not real sentences, they have been shown to act as a regularization method which can make neural networks more robust.

Built on

Y. LeCun, Y. Bengio et al. , “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks , vol. 3361, no. 10, p. 1995, 1995
1995
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation , vol. 9, no. 8, pp. 1735–1780, 1997
1997
Earlier work this paper cites.
K. Nigam, J. Lafferty, and A. McCallum, “Using maximum entropy for text classification,” in IJCAI-99 workshop on machine learning for information filtering , vol. 1, no. 1, 1999, pp. 61–67
1999
Earlier work this paper cites.
M. Hu and B. Liu, “Mining and summarizing customer reviews,” in Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining . ACM, 2004, pp. 168–177
2004
Earlier work this paper cites.
M. Pontiki, D. Galanis, J. Pavlopoulos, H. Papageorgiou, I. Androutsopoulos, and S. Manandhar, “SemEval-2014 task 4: Aspect based sentiment analysis,” in Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) . Dublin, Ireland: Association for Computational Linguistics, Aug. 2014, pp. 27–35. [Online]. Available: https://www.aclweb.org/anthology/S14-2004
2004
Earlier work this paper cites.
I. Titov and R. McDonald, “A joint model of text and aspect ratings for sentiment summarization,” in proceedings of ACL-08: HLT , 2008, pp. 308–316
2008
Earlier work this paper cites.
T. T. Thet, J.-C. Na, and C. S. Khoo, “Aspect-based sentiment analysis of movie reviews on discussion boards,” Journal of information science , vol. 36, no. 6, pp. 823–848, 2010
2010
Earlier work this paper cites.
S. Wakade, C. Shekar, K. J. Liszka, and C.-C. Chan, “Text mining for sentiment analysis of twitter data,” in Proceedings of the International Conference on Information and Knowledge Engineering (IKE) . The Steering Committee of The World Congress in Computer Science, Computer …, 2012, p. 1
2012
Earlier work this paper cites.
I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572 , 2014
Original
2014
Earlier work this paper cites.
S. Kiritchenko, X. Zhu, C. Cherry, and S. Mohammad, “Nrc-canada-2014: Detecting aspects and sentiment in customer reviews,” in Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014) , 2014, pp. 437–442
2014
Earlier work this paper cites.
P. Gamallo and M. Garcia, “Citius: A naive-bayes strategy for sentiment analysis on english tweets,” in Proceedings of the 8th international Workshop on Semantic Evaluation (SemEval 2014) , 2014, pp. 171–175
2014
Earlier work this paper cites.
Y. Kim, “Convolutional neural networks for sentence classification,” in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) , 2014, pp. 1746–1751
2014
Earlier work this paper cites.

Similar

C. Szegedy, W. Zaremba, I. Sutskever, J. B. Estrach, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in 2nd International Conference on Learning Representations, ICLR 2014 , 2014
2014
Cited alongside, same era.
D. Bahdanau, K. Cho, and Y. Bengio, “Neural machine translation by jointly learning to align and translate,” arXiv preprint arXiv:1409.0473 , 2014
Original
2014
Cited alongside, same era.
X. Zhang, J. Zhao, and Y. LeCun, “Character-level convolutional networks for text classification,” in Advances in neural information processing systems , 2015, pp. 649–657
2015
Cited alongside, same era.
P. Liu, S. Joty, and H. Meng, “Fine-grained opinion mining with recurrent neural networks and word embeddings,” in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing , 2015, pp. 1433–1443
2015
Cited alongside, same era.
Y. Wang, M. Huang, L. Zhao et al. , “Attention-based lstm for aspect-level sentiment classification,” in Proceedings of the 2016 conference on empirical methods in natural language processing , 2016, pp. 606–615
2016
Cited alongside, same era.
T. Miyato, A. M. Dai, and I. Goodfellow, “Adversarial training methods for semi-supervised text classification,” arXiv preprint arXiv:1605.07725 , 2016
Original
2016
Cited alongside, same era.
S. Poria, E. Cambria, and A. Gelbukh, “Aspect extraction for opinion mining with a deep convolutional neural network,” Knowledge-Based Systems , vol. 108, pp. 42–49, 2016
2016
Cited alongside, same era.
Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi, W. Macherey, M. Krikun, Y. Cao, Q. Gao, K. Macherey et al. , “Google’s neural machine translation system: Bridging the gap between human and machine translation,” arXiv preprint arXiv:1609.08144 , 2016
Original
2016
Cited alongside, same era.
M. Pontiki, D. Galanis, H. Papageorgiou, I. Androutsopoulos, S. Manandhar, A.-S. Mohammad, M. Al-Ayyoub, Y. Zhao, B. Qin, O. De Clercq et al. , “Semeval-2016 task 5: Aspect based sentiment analysis,” in Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016) , 2016, pp. 19–30
2016
Cited alongside, same era.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in neural information processing systems , 2017, pp. 5998–6008
2017
Cited alongside, same era.
R. He, W. S. Lee, H. T. Ng, and D. Dahlmeier, “An unsupervised neural attention model for aspect extraction,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2017, pp. 388–397
2017
Cited alongside, same era.
Y. Ma, H. Peng, and E. Cambria, “Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm,” in Thirty-Second AAAI Conference on Artificial Intelligence , 2018
2018
Cited alongside, same era.

Then

A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. Bowman, “Glue: A multi-task benchmark and analysis platform for natural language understanding,” in Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP , 2018, pp. 353–355
2018
Later among the works it cites.
H. Xu, B. Liu, L. Shu, and S. Y. Philip, “Double embeddings and cnn-based sequence labeling for aspect extraction,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , 2018, pp. 592–598
2018
Later among the works it cites.
X. Li, L. Bing, P. Li, W. Lam, and Z. Yang, “Aspect term extraction with history attention and selective transformation,” arXiv preprint arXiv:1805.00760 , 2018
Original
2018
Later among the works it cites.
J. Ebrahimi, A. Rao, D. Lowd, and D. Dou, “Hotflip: White-box adversarial examples for text classification,” in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , 2018, pp. 31–36
2018
Later among the works it cites.
A. Ilyas, L. Engstrom, A. Athalye, and J. Lin, “Black-box adversarial attacks with limited queries and information,” in International Conference on Machine Learning , 2018, pp. 2137–2146
2018
Later among the works it cites.
H. Xu, B. Liu, L. Shu, and S. Y. Philip, “Bert post-training for review reading comprehension and aspect-based sentiment analysis,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , 2019, pp. 2324–2335
2019
Later among the works it cites.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) , 2019, pp. 4171–4186
2019
Later among the works it cites.
Z. Li, Y. Wei, Y. Zhang, X. Zhang, and X. Li, “Exploiting coarse-to-fine task transfer for aspect-level sentiment classification,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 33, 2019, pp. 4253–4260
2019
Later among the works it cites.
Y. Song, J. Wang, T. Jiang, Z. Liu, and Y. Rao, “Attentional encoder network for targeted sentiment classification,” arXiv preprint arXiv:1902.09314 , 2019
Original
2019
Later among the works it cites.
C. Zhu, Y. Cheng, Z. Gan, S. Sun, T. Goldstein, and J. Liu, “Freelb: Enhanced adversarial training for natural language understanding,” in International Conference on Learning Representations , 2019
2019
Later among the works it cites.
A. Rietzler, S. Stabinger, P. Opitz, and S. Engl, “Adapt or get left behind: Domain adaptation through bert language model finetuning for aspect-target sentiment classification,” in Proceedings of The 12th Language Resources and Evaluation Conference , 2020, pp. 4933–4941
2020
Closest in time.
P. Zhao, L. Hou, and O. Wu, “Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification,” Knowledge-Based Systems , vol. 193, p. 105443, 2020
2020
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…