2020

Dos and Don'ts of Machine Learning in Computer Security

Arp, Daniel, Quiring, Erwin, Pendlebury, Feargus et al.

Understand

With the growing processing power of computing systems and the increasing availability of massive datasets, machine learning algorithms have led to major breakthroughs in many different areas.

This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis.
Despite great potential, machine learning in security is prone to subtle pitfalls that undermine its performance and render learning-based systems potentially unsuitable for security tasks and practical deployment.
In this paper, we look at this problem with critical eyes.

Built on

Comparison of the predicted and observed secondary structure of t4 phage lysozyme
B. Matthews · 1975
Earlier work this paper cites.
Exploratory data analysis
J. W. Tukey · 1977
Earlier work this paper cites.
Sample Selection Bias as a Specification Error
J. J. Heckman · 1979
Earlier work this paper cites.
An intrusion-detection model
D. E. Denning · 1986
Earlier work this paper cites.
A sense of self for unix processes
S. Forrest, S. A. Hofmeyr, A. Somayaji, and T. A. Longstaff · 1996
Earlier work this paper cites.
The lack of a priori distinctions between learning algorithms
D. H. Wolpert · 1996
Earlier work this paper cites.
Long short-term memory
S. Hochreiter and J. Schmidhuber · 1997
Earlier work this paper cites.
Data mining approaches for intrusion detection
W. Lee and S. J. Stolfo · 1998
Earlier work this paper cites.
Bro: A system for detecting network intruders in real-time
V. Paxson · 1998
Earlier work this paper cites.
Snort - lightweight intrusion detection for networks
M. Roesch · 1999
Earlier work this paper cites.
The base-rate fallacy and the difficulty of intrusion detection
S. Axelsson · 2000
Earlier work this paper cites.
Testing intrusion detection systems: A critique of the 1998 and 1999 darpa intrusion detection system evaluations as performed by lincoln laboratory
J. McHugh · 2000
Earlier work this paper cites.
SMOTE: synthetic minority over-sampling technique
N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer · 2002
Earlier work this paper cites.
"why 6?" defining the operational limits of stide, an anomaly-based intrusion detector
K. M. C. Tan and R. A. Maxion · 2002
Earlier work this paper cites.
A pitfall and solution in multi-class feature selection for text classification
G. Forman · 2004
Earlier work this paper cites.
Learning and evaluating classifiers under sample selection bias
B. Zadrozny · 2004
Earlier work this paper cites.
Borderline-smote: A new over-sampling method in imbalanced data sets learning
H. Han, W.-Y. Wang, and B.-H. Mao · 2005
Earlier work this paper cites.
A framework for the evaluation of intrusion detection systems
A. A. Cardenas, J. S. Baras, and K. Seamon · 2006
Earlier work this paper cites.
The relationship between precision-recall and roc curves
J. Davis and M. Goadrich · 2006
Earlier work this paper cites.
Anagram: A content anomaly detector resistant to mimicry attack
K. Wang, J. J. Parekh, and S. J. Stolfo · 2006
Earlier work this paper cites.
Sample selection bias correction theory
C. Cortes, M. Mohri, M. Riley, and A. Rostamizadeh · 2008
Earlier work this paper cites.
LIBLINEAR: A library for large linear classification
R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin · 2008
Earlier work this paper cites.
Anomaly detection: A survey
V. Chandola, A. Banerjee, and V. Kumar · 2009
Earlier work this paper cites.
On lightweight mobile phone application certification
W. Enck, M. Ongtang, and P. D. McDaniel · 2009
Earlier work this paper cites.
Measuring Classifier Performance: a Coherent Alternative to the Area Under the ROC Curve
D. J. Hand · 2009
Earlier work this paper cites.
Protecting a moving target: Addressing web application concept drift
F. Maggi, W. Robertson, C. Kruegel, and G. Vigna · 2009
Earlier work this paper cites.
Zero-shot learning with semantic output codes
M. Palatucci, D. Pomerleau, G. Hinton, and T. M. Mitchell · 2009
Earlier work this paper cites.
A systematic analysis of performance measures for classification tasks
M. Sokolova and G. Lapalme · 2009
Earlier work this paper cites.
Apples-to-apples in cross-validation studies: Pitfalls in classifier performance measurement
G. Forman and M. Scholz · 2010
Earlier work this paper cites.
Synthesizing near-optimal malware specifications from suspicious behaviors
M. Fredrikson, S. Jha, M. Christodorescu, R. Sailer, and X. Yan · 2010
Earlier work this paper cites.
Outside the closed world: On using machine learning for network intrusion detection
R. Sommer and V. Paxson · 2010
Earlier work this paper cites.
Support vector machines under adversarial label noise
B. Biggio, B. Nelson, and P. Laskov · 2011
Earlier work this paper cites.
Zozzle: Fast and precise in-browser javascript malware detection
C. Curtsinger, B. Livshits, B. Zorn, and C. Seifert · 2011
Earlier work this paper cites.
Bitshred: feature hashing malware for scalable triage and semantic analysis
J. Jang, D. Brumley, and S. Venkataraman · 2011
Earlier work this paper cites.
Unbiased look at dataset bias
A. Torralba and A. A. Efros · 2011
Earlier work this paper cites.
Learning From Data , chapter 5
Y. S. Abu-Mostafa, M. Magdon-Ismail, and H.-T. Lin · 2012
Earlier work this paper cites.
Peek-a-boo, I still see you: Why efficient traffic analysis countermeasures fail
K. P. Dyer, S. E. Coull, T. Ristenpart, and T. Shrimpton · 2012
Earlier work this paper cites.
ImageNet: Classification with deep convolutional neural networks
A. Krizhevsky, I. Sutskever, and G. E. Hinton · 2012
Earlier work this paper cites.
An efficient intrusion detection system based on support vector machines and gradually feature removal method
Y. Li, J. Xia, S. Zhang, J. Yan, X. Ai, and K. Dai · 2012
Earlier work this paper cites.
A unifying view on dataset shift in classification
J. G. Moreno-Torres, T. Raeder, R. Alaiz-RodríGuez, N. V. Chawla, and F. Herrera · 2012
Earlier work this paper cites.
Prudent practices for designing malware experiments: Status quo and outlook
C. Rossow, C. Dietrich, C. Gier, C. Kreibich, V. Paxson, N. Pohlmann, H. Bos, and M. van Steen · 2012
Earlier work this paper cites.
Evasion attacks against machine learning at test time
B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndić, P. Laskov, G. Giacinto, and F. Roli · 2013
Earlier work this paper cites.
Detection of malicious PDF files based on hierarchical document structure
N. Srndic and P. Laskov · 2013
Earlier work this paper cites.
Drebin: Efficient and explainable detection of Android malware in your pocket
D. Arp, M. Spreitzenbarth, M. Hübner, H. Gascon, and K. Rieck · 2014
Earlier work this paper cites.
BYTEWEIGHT: Learning to recognize functions in binary code
T. Bao, J. Burket, M. Woo, R. Turner, and D. Brumley · 2014
Earlier work this paper cites.
Recovering from selection bias in causal and statistical inference
E. Bareinboim, J. Tian, and J. Pearl · 2014
Earlier work this paper cites.
Security evaluation of pattern classifiers under attack
B. Biggio, G. Fumera, and F. Roli · 2014
Earlier work this paper cites.
Learning phrase representations using RNN encoder-decoder for statistical machine translation
K. Cho, B. van Merrienboer, Ç. Gülçehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio · 2014
Earlier work this paper cites.

Similar

Classification in the presence of label noise: A survey
B. Frenay and M. Verleysen · 2014
Cited alongside, same era.
A critical evaluation of website fingerprinting attacks
M. Juarez, S. Afroz, G. Acar, C. Diaz, and R. Greenstadt · 2014
Cited alongside, same era.
Robust classification under sample selection bias
A. Liu and B. D. Ziebart · 2014
Cited alongside, same era.
Sequence to sequence learning with neural networks
I. Sutskever, O. Vinyals, and Q. V. Le · 2014
Cited alongside, same era.
Practical evasion of a learning-based classifier: A case study
N. Šrndić and P. Laskov · 2014
Cited alongside, same era.
Modeling and discovering vulnerabilities with code property graphs
F. Yamaguchi, N. Golde, D. Arp, and K. Rieck · 2014
Cited alongside, same era.
Are your training datasets yet relevant? - an investigation into the importance of timeline in machine learning-based malware detection
K. Allix, T. F. Bissyandé, J. Klein, and Y. L. Traon · 2015
Cited alongside, same era.
On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation
S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek · 2015
Cited alongside, same era.
Neural machine translation by jointly learning to align and translate
D. Bahdanau, K. Cho, and Y. Bengio · 2015
Cited alongside, same era.
Integro: Leveraging victim prediction for robust fake account detection in osns
Y. Boshmaf, D. Logothetis, G. Siganos, J. Lería, J. Lorenzo, M. Ripeanu, and K. Beznosov · 2015
Cited alongside, same era.
De-anonymizing programmers via code stylometry
A. Caliskan, R. Harang, A. Liu, A. Narayanan, C. R. Voss, F. Yamaguchi, and R. Greenstadt · 2015
Cited alongside, same era.
Efficient and robust automated machine learning
M. Feurer, A. Klein, K. Eggensperger, J. T. Springenberg, M. Blum, and F. Hutter · 2015
Cited alongside, same era.

Then

Lemna: Explaining deep learning based security applications
W. Guo, D. Mu, J. Xu, P. Su, G. Wang, and X. Xing · 2018
Later among the works it cites.
Surveylance: Automatically detecting online survey scams
A. Kharraz, W. K. Robertson, and E. Kirda · 2018
Later among the works it cites.
A simple way to deal with cherry-picking
Original
J. Komiyama and T. Maehara · 2018
Later among the works it cites.
Content analysis: An introduction to its methodology
K. Krippendorff · 2018
Later among the works it cites.
Vuldeepecker: A deep learning-based system for vulnerability detection
Z. Li, D. Zou, S. Xu, X. Ou, H. Jin, S. Wang, Z. Deng, and Y. Zhong · 2018
Later among the works it cites.
Detecting and correcting for label shift with black box predictors
Z. C. Lipton, Y. Wang, and A. J. Smola · 2018
Later among the works it cites.
Kitsune: An ensemble of autoencoders for online network intrusion detection
Y. Mirsky, T. Doitshman, Y. Elovici, and A. Shabtai · 2018
Later among the works it cites.
SoK: Security and privacy in machine learning
N. Papernot, P. McDaniel, A. Sinha, and M. P. Wellman · 2018
Later among the works it cites.
Automated website fingerprinting through deep learning
V. Rimmer, D. Preuveneers, M. Juárez, T. van Goethem, and W. Joosen · 2018
Later among the works it cites.
Tiresias: Predicting security events through deep learning
Y. Shen, E. Mariconti, P. Vervier, and G. Stringhini · 2018
Later among the works it cites.
Convnets and imagenet beyond accuracy: Understanding mistakes and uncovering biases
P. Stock and M. Cissé · 2018
Later among the works it cites.
On evaluating adversarial robustness
Original
N. Carlini, A. Athalye, N. Papernot, W. Brendel, J. Rauber, D. Tsipras, I. J. Goodfellow, A. Madry, and A. Kurakin · 2019
Later among the works it cites.
Sok: The challenges, pitfalls, and perils of using hardware performance counters for security
S. Das, J. Werner, M. Antonakakis, M. Polychronakis, and F. Monrose · 2019
Later among the works it cites.
Yes, machine learning can be more secure! a case study on android malware detection
A. Demontis, M. Melis, B. Biggio, D. Maiorca, D. Arp, K. Rieck, I. Corona, G. Giacinto, and F. Roli · 2019
Later among the works it cites.
Asm2vec: Boosting static representation robustness for binary clone search against code obfuscation and compiler optimization
S. H. H. Ding, B. C. M. Fung, and P. Charland · 2019
Later among the works it cites.
Stack overflow considered helpful! deep learning security nudges towards stronger cryptography
F. Fischer, H. Xiao, C.-Y. Kao, Y. Stachelscheid, B. Johnson, D. Razar, P. Fawkesley, N. Buckley, K. Böttinger, P. Muntean, and J. Grossklags · 2019
Later among the works it cites.
A benchmark for interpretability methods in deep neural networks
S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim · 2019
Later among the works it cites.
Auto-keras: An efficient neural architecture search system
H. Jin, Q. Song, and X. Hu · 2019
Later among the works it cites.
The (un)reliability of saliency methods
P.-J. Kindermans, S. Hooker, J. Adebayo, M. Alber, K. T. Schütt, S. Dähne, D. Erhan, and B. Kim · 2019
Later among the works it cites.
Unmasking Clever Hans predictors and assessing what machines really learn
S. Lapuschkin, S. Wäldchen, A. Binder, G. Montavon, W. Samek, and K.-R. Müller · 2019
Later among the works it cites.
Rethinking location privacy for unknown mobility behaviors
S. Oya, C. Troncoso, and F. Pérez-González · 2019
Later among the works it cites.
TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time
F. Pendlebury, F. Pierazzi, R. Jordaney, J. Kinder, and L. Cavallaro · 2019
Later among the works it cites.
Misleading authorship attribution of source code using adversarial learning
E. Quiring, A. Maier, and K. Rieck · 2019
Later among the works it cites.
Robust website fingerprinting through the cache occupancy channel
A. Shusterman, L. Kang, Y. Haskal, Y. Meltser, P. Mittal, Y. Oren, and Y. Yarom · 2019
Later among the works it cites.
Robust performance metrics for authentication systems
S. Sugrim, C. Liu, M. McLean, and J. Lindqvist · 2019
Later among the works it cites.
SoK: Benchmarking Flaws in Systems Security
E. van der Kouwe, G. Heiser, D. Andriesse, H. Bos, and C. Giuffrida · 2019
Later among the works it cites.
Deepintent: Deep icon-behavior learning for detecting intention-behavior discrepancy in mobile apps
S. Xi, S. Yang, X. Xiao, Y. Yao, Y. Xiong, F. Xu, H. Wang, P. Gao, Z. Liu, F. Xu, and J. Lu · 2019
Later among the works it cites.
When Malware is Packin’ Heat; Limits of Machine Learning Classifiers Based on Static Analysis Features
H. Aghakhani, F. Gritti, F. Mecca, M. Lindorfer, S. Ortolani, D. Balzarotti, G. Vigna, and C. Kruegel · 2020
Closest in time.
Transcending transcend: Revisiting malware classification with conformal evaluation
Original
F. Barbero, F. Pendlebury, F. Pierazzi, and L. Cavallaro · 2020
Closest in time.
The advantages of the matthews correlation coefficient (mcc) over f1 score and accuracy in binary classification evaluation
D. Chicco and G. Jurman · 2020
Closest in time.
https://nvd.nist.gov/
N. V. Database · 2020
Closest in time.
https://samate.nist.gov/SRD/index.php
S. A. R. Dataset · 2020
Closest in time.
An overview of privacy in machine learning
Original
E. De Cristofaro · 2020
Closest in time.
AAAS: Machine learning ’causing science crisis’
P. Ghosh · 2020
Closest in time.
Adgraph: A graph-based approach to ad and tracker blocking
U. Iqbal, P. Snyder, S. Zhu, B. Livshits, Z. Qian, and Z. Shafiq · 2020
Closest in time.
Correcting for selection bias in learning-to-rank systems
Z. Ovaisi, R. Ahsan, Y. Zhang, K. Vasilaky, and E. Zheleva · 2020
Closest in time.
Intriguing properties of adversarial ml attacks in the problem space
F. Pierazzi, F. Pendlebury, J. Cortellazzi, and L. Cavallaro · 2020
Closest in time.
Adversarial preprocessing: Understanding and preventing image-scaling attacks in machine learning
E. Quiring, D. Klein, D. Arp, M. Johns, and K. Rieck · 2020
Closest in time.
Sanity checks for saliency metrics
R. Tomsett, D. Harborne, S. Chakraborty, P. Gurram, and A. Preece · 2020
Closest in time.
Evaluating explanation methods for deep learning in security
A. Warnecke, D. Arp, C. Wressnegger, and K. Rieck · 2020
Closest in time.
Troubleshooting an intrusion detection dataset: the CICIDS2017 case study
G. Engelen, V. Rimmer, and W. Joosen · 2021
Closest in time.
FARE: Enabling Fine-grained Attack Categorization under Low-quality Labeled Data
J. Liang, W. Guo, T. Luo, V. Honavar, G. Wang, and X. Xing · 2021
Closest in time.
Confident learning: Estimating uncertainty in dataset labels
C. G. Northcutt, L. Jiang, and I. L. Chuang · 2021
Closest in time.
difflib – helpers for computing deltas
Python Software Foundation · 2021
Closest in time.
A framework for understanding sources of harm throughout the machine learning life cycle
Original
H. Suresh and J. V. Guttag · 2021
Closest in time.
Differential training: A generic framework to reduce label noises for android malware detection
J. Xu, Y. Li, and R. H. Deng · 2021
Closest in time.
A comprehensive survey on transfer learning
F. Zhuang, Z. Qi, K. Duan, D. Xi, Y. Zhu, H. Zhu, H. Xiong, and Q. He · 2021
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…