2020

Text Data Augmentation: Towards better detection of spear-phishing emails

Regina, Mehdi, Meyer, Maxime, Goutal, Sébastien

Understand

Text data augmentation, i.e., the creation of new textual data from an existing text, is challenging.

Indeed, augmentation transformations should take into account language complexity while being relevant to the target Natural Language Processing (NLP) task (e.g., Machine Translation, Text Classification).
Initially motivated by an application of Business Email Compromise (BEC) detection, we propose a corpus and task agnostic augmentation framework used as a service to augment English texts within our company.
Our proposal combines different methods, utilizing BERT language model, multi-step back-translation and heuristics.

Built on

McCarthy, J.: Applications of circumscription to formalizing common-sense knowledge. Artificial intelligence 28
1986
Earlier work this paper cites.
Miller, G.A.: WordNet: An electronic lexical database. MIT press (1998)
1998
Earlier work this paper cites.
Li, X., Roth, D.: Learning question classifiers. In: Proceedings of the 19th international conference on Computational linguistics-Volume 1. pp. 1–7. Association for Computational Linguistics (2002)
2002
Earlier work this paper cites.
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting on association for computational linguistics. pp. 311–318. Association for Computational Linguistics (2002)
2002
Earlier work this paper cites.
Klimt, B., Yang, Y.: Introducing the enron corpus. In: CEAS (2004)
2004
Earlier work this paper cites.
Dolan, W.B., Brockett, C.: Automatically constructing a corpus of sentential paraphrases. In: Proceedings of the Third International Workshop on Paraphrasing (IWP2005) (2005)
2005
Earlier work this paper cites.
Li, Y., McLean, D., Bandar, Z.A., O’shea, J.D., Crockett, K.: Sentence similarity based on semantic nets and corpus statistics. IEEE transactions on knowledge and data engineering 18
2006
Earlier work this paper cites.
Mihalcea, R., Corley, C., Strapparava, C., et al.: Corpus-based and knowledge-based measures of text semantic similarity. In: Aaai. vol. 6, pp. 775–780 (2006)
2006
Earlier work this paper cites.
Idika, N., Mathur, A.P.: A survey of malware detection techniques. Purdue University 48
2007
Earlier work this paper cites.
Howard, F.: Modern web attacks. Network Security 2008
2008
Earlier work this paper cites.
Huang, A.: Similarity measures for text document clustering. In: Proceedings of the sixth new zealand computer science research student conference (NZCSRSC2008), Christchurch, New Zealand. vol. 4, pp. 9–56 (2008)
2008
Earlier work this paper cites.
Maaten, L.v.d., Hinton, G.: Visualizing data using t-sne. Journal of machine learning research 9
2008
Earlier work this paper cites.
Oliver, J., Eikenberry, S.D., Budman, G., Kim, B.: Fraudulent message detection (Nov 11 2008), uS Patent 7,451,487
2008
Earlier work this paper cites.
Levow, Z., Drako, D.: Multilevel intent analysis method for email filtration (Dec 3 2009), uS Patent App. 12/128,286
2009
Earlier work this paper cites.
Stanojević, M., et al.: Cognitive synonymy: A general overview. FACTA UNIVERSITATIS-Linguistics and Literature 7
2009
Earlier work this paper cites.
Warner, W., Hirschberg, J.: Detecting hate speech on the world wide web. In: Proceedings of the Second Workshop on Language in Social Media. pp. 19–26. LSM ’12, Association for Computational Linguistics (2012)
2012
Earlier work this paper cites.
Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2013)
2013
Earlier work this paper cites.
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 conference on empirical methods in natural language processing. pp. 1631–1642 (2013)
2013
Earlier work this paper cites.
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Original
2014
Earlier work this paper cites.
Rezende, D.J., Mohamed, S., Wierstra, D.: Stochastic backpropagation and approximate inference in deep generative models (2014)
2014
Earlier work this paper cites.

Similar

Jin, J., Dundar, A., Culurciello, E.: Robust convolutional neural networks under adversarial noise. arXiv preprint arXiv:1511.06306 (2015)
Original
2015
Cited alongside, same era.
Nissim, N., Cohen, A., Glezer, C., Elovici, Y.: Detection of malicious pdf files and directions for enhancements: A state-of-the art survey. Computers & Security 48
2015
Cited alongside, same era.
Sennrich, R., Haddow, B., Birch, A.: Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709 (2015)
Original
2015
Cited alongside, same era.
Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huck, M., Yepes, A.J., Koehn, P., Logacheva, V., Monz, C., et al.: Findings of the 2016 conference on machine translation. In: Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers. pp. 131–198 (2016)
2016
Cited alongside, same era.
Jakobsson, M.: Understanding social engineering based scams. Springer (2016)
2016
Cited alongside, same era.
Krawczyk, B.: Learning from imbalanced data: open challenges and future directions. Progress in Artificial Intelligence 5
2016
Cited alongside, same era.
Mansfield-Devine, S.: The imitation game: How business email compromise scams are robbing organisations. Computer Fraud & Security 2016
2016
Cited alongside, same era.
Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al.: Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)
Original
2016
Cited alongside, same era.
Conneau, A., Kiela, D., Schwenk, H., Barrault, L., Bordes, A.: Supervised learning of universal sentence representations from natural language inference data. arXiv preprint arXiv:1705.02364 (2017)
Original
2017
Cited alongside, same era.
Ho, G., Sharma, A., Javed, M., Paxson, V., Wagner, D.: Detecting credential spearphishing in enterprise settings. In: 26th { \{ USENIX } \} Security Symposium ( { \{ USENIX } \} Security 17). pp. 469–485 (2017)
2017
Cited alongside, same era.
Li, Z., Jiang, X., Shang, L., Li, H.: Paraphrase generation with deep reinforcement learning. arXiv preprint arXiv:1711.00279 (2017)
Original
2017
Cited alongside, same era.
Perez, L., Wang, J.: The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621 (2017)
Original
2017
Cited alongside, same era.

Then

Kobayashi, S.: Contextual augmentation: Data augmentation by words with paradigmatic relations. arXiv preprint arXiv:1805.06201 (2018)
Original
2018
Later among the works it cites.
Proofpoint: Understanding email fraud. Tech. rep. (03 2018)
2018
Later among the works it cites.
Zhang, D., Yang, Z.: Word embedding perturbation for sentence classification. arXiv preprint arXiv:1804.08166 (2018)
Original
2018
Later among the works it cites.
Cidon, A., Gavish, L., Bleier, I., Korshun, N., Schweighauser, M., Tsitkin, A.: High precision detection of business email compromise. In: 28th { \{ USENIX } \} Security Symposium ( { \{ USENIX } \} Security 19). pp. 1291–1307 (2019)
2019
Later among the works it cites.
Giridhara, P.K.B., Chinmaya, M., Venkataramana, R.K.M., Bukhari, S.S., Dengel, A.: A study of various text augmentation techniques for relation classification in free text. In: Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods (2019)
2019
Later among the works it cites.
Holtzman, A., Buys, J., Du, L., Forbes, M., Choi, Y.: The curious case of neural text degeneration. arXiv preprint arXiv:1904.09751 (2019)
Original
2019
Later among the works it cites.
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., Zettlemoyer, L.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
Original
2019
Later among the works it cites.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. OpenAI Blog 1
2019
Later among the works it cites.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019)
Original
2019
Later among the works it cites.
Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)
Original
2019
Later among the works it cites.
Wei, J.W., Zou, K.: Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv preprint arXiv:1901.11196 (2019)
Original
2019
Later among the works it cites.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al.: Huggingface’s transformers: State-of-the-art natural language processing. ArXiv, abs/1910.03771 (2019)
Original
2019
Later among the works it cites.
Wu, X., Lv, S., Zang, L., Han, J., Hu, S.: Conditional bert contextual augmentation. In: International Conference on Computational Science. pp. 84–95. Springer (2019)
2019
Later among the works it cites.
Xie, Q., Dai, Z., Hovy, E., Luong, M.T., Le, Q.V.: Unsupervised data augmentation. arXiv preprint arXiv:1904.12848 (2019)
Original
2019
Later among the works it cites.
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R.R., Le, Q.V.: Xlnet: Generalized autoregressive pretraining for language understanding. In: Advances in neural information processing systems. pp. 5754–5764 (2019)
2019
Later among the works it cites.
Zhang, Y., Baldridge, J., He, L.: Paws: Paraphrase adversaries from word scrambling. arXiv preprint arXiv:1904.01130 (2019)
Original
2019
Later among the works it cites.
Kumar, V., Choudhary, A., Cho, E.: Data augmentation using pre-trained transformer models. arXiv preprint arXiv:2003.02245 (2020)
Original
2020
Closest in time.
Kwak, Y., Lee, S., Damiano, A., Vishwanath, A.: Why do users not report spear phishing emails? Telematics and Informatics p. 101343 (2020)
2020
Closest in time.
FBI: 2020 internet crime report. Tech. rep., Federal Bureau of Investigation, Internet Crime Complaint Center (03 2021)
2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…