2017

Unsupervised and Semi-supervised Anomaly Detection with LSTM Neural Networks

Ergen, Tolga, Mirza, Ali Hassan, Kozat, Suleyman Serdar

Understand

We investigate anomaly detection in an unsupervised framework and introduce Long Short Term Memory (LSTM) neural network based algorithms.

In particular, given variable length data sequences, we first pass these sequences through our LSTM based structure and obtain fixed length sequences.
We then find a decision function for our anomaly detectors based on the One Class Support Vector Machines (OC-SVM) and Support Vector Data Description (SVDD) algorithms.
As the first time in the literature, we jointly train and optimize the parameters of the LSTM architecture and the OC-SVM (or SVDD) algorithm using highly effective gradient and quadratic programming based training methods.

Built on

L. Veidinger, “On the numerical determination of the best approximations in the chebyshev sense,” Numerische Mathematik , vol. 2, no. 1, pp. 99–105, 1960
1960
Earlier work this paper cites.
N. H. Packard, J. P. Crutchfield, J. D. Farmer, and R. S. Shaw, “Geometry from a time series,” Physical review letters , vol. 45, no. 9, p. 712, 1980
1980
Earlier work this paper cites.
H. Debar, M. Becker, and D. Siboni, “A neural network component for an intrusion detection system,” in Proceedings 1992 IEEE Computer Society Symposium on Research in Security and Privacy , May 1992, pp. 240–250
1992
Earlier work this paper cites.
Y. Bengio, P. Simard, and P. Frasconi, “Learning long-term dependencies with gradient descent is difficult,” IEEE Transactions on Neural Networks , vol. 5, no. 2, pp. 157–166, Mar 1994
1994
Earlier work this paper cites.
R. Kozma et al. , “Anomaly detection by neural network models and statistical time series analysis,” in Neural Networks, 1994. IEEE World Congress on Computational Intelligence., 1994 IEEE International Conference on , vol. 5, Jun 1994, pp. 3207–3210 vol.5
1994
Earlier work this paper cites.
C. Bishop, “Novelty detection and neural network validation,” IEE Proceedings - Vision, Image and Signal Processing , vol. 141, pp. 217–222(5), August 1994
1994
Earlier work this paper cites.
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput. , vol. 9, no. 8, pp. 1735–1780, Nov. 1997
1997
Earlier work this paper cites.
A. P. Bradley, “The use of the area under the roc curve in the evaluation of machine learning algorithms,” Pattern Recognition , vol. 30, no. 7, pp. 1145 – 1159, 1997
1997
Earlier work this paper cites.
J. Platt, “Sequential minimal optimization: A fast algorithm for training support vector machines,” 1998
1998
Earlier work this paper cites.
K. P. Bennett and A. Demiriz, “Semi-supervised support vector machines,” in Advances in Neural Information processing systems , 1999, pp. 368–374
1999
Earlier work this paper cites.
B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, “Estimating the support of a high-dimensional distribution,” Neural computation , vol. 13, no. 7, pp. 1443–1471, 2001
2001
Earlier work this paper cites.
J. Ma and S. Perkins, “Time-series novelty detection using one-class support vector machines,” in Proceedings of the International Joint Conference on Neural Networks, 2003. , vol. 3, July 2003, pp. 1741–1745 vol.3
2003
Earlier work this paper cites.

Similar

A. H. Sayed, Fundamentals of adaptive filtering . John Wiley & Sons, 2003
2003
Cited alongside, same era.
D. M. Tax and R. P. Duin, “Support vector data description,” Machine Learning , vol. 54, no. 1, pp. 45–66, 2004. [Online]. Available: http://dx.doi.org/10.1023/B:MACH.0000008084.60811.49
2004
Cited alongside, same era.
Y. Wang, J. Wong, and A. Miner, “Anomaly intrusion detection using one class svm,” in Information Assurance Workshop, 2004. Proceedings from the Fifth Annual IEEE SMC . IEEE, 2004, pp. 358–364
2004
Cited alongside, same era.
R. Zhang et al. , “One class support vector machine for anomaly detection in the communication network performance data,” in Proceedings of the 5th Conference on Applied Electromagnetics, Wireless and Optical Communications , ser. ELECTROSCIENCE’07. Stevens Point, Wisconsin, USA: World Scientific and Engineering Academy and Society (WSEAS), 2007, pp. 31–37
2007
Cited alongside, same era.
R. Zhang, S. Zhang, Y. Lan, and J. Jiang, “Network anomaly detection using one class support vector machine,” in Proceedings of the International MultiConference of Engineers and Computer Scientists , vol. 1, 2008
2008
Cited alongside, same era.
V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM Comput. Surv. , vol. 41, no. 3, pp. 15:1–15:58, Jul. 2009. [Online]. Available: http://doi.acm.org/10.1145/1541880.1541882
2009
Cited alongside, same era.
C.-C. Chang and C.-J. Lin, “LIBSVM: A library for support vector machines,” ACM Transactions on Intelligent Systems and Technology , vol. 2, pp. 27:1–27:27, 2011, software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
2011
Cited alongside, same era.
Z. Wen and W. Yin, “A feasible method for optimization with orthogonality constraints,” Mathematical Programming , vol. 142, no. 1, pp. 397–434, Dec 2013
2013
Cited alongside, same era.
N. Görnitz, M. Kloft, K. Rieck, and U. Brefeld, “Toward supervised anomaly detection,” J. Artif. Int. Res. , vol. 46, no. 1, pp. 235–262, Jan. 2013. [Online]. Available: http://dl.acm.org/citation.cfm?id=2512538.2512545
2013
Cited alongside, same era.
M. Lichman, “UCI machine learning repository,” 2013
2013
Cited alongside, same era.
K. Cohen, Q. Zhao, and A. Swami, “Optimal index policies for anomaly localization in resource-constrained cyber systems,” IEEE Transactions on Signal Processing , vol. 62, no. 16, pp. 4224–4236, Aug 2014
2014
Cited alongside, same era.
J. Chung, C. Gulcehre, K. Cho, and Y. Bengio, “Empirical evaluation of gated recurrent neural networks on sequence modeling,” arXiv preprint arXiv:1412.3555 , 2014
Original
2014
Cited alongside, same era.

Then

P. Malhotra, L. Vig, G. Shroff, and P. Agarwal, “Long short term memory networks for anomaly detection in time series,” in Proceedings . Presses universitaires de Louvain, 2015, p. 89
2015
Later among the works it cites.
S. Chauhan and L. Vig, “Anomaly detection in ecg time signals via deep long short-term memory networks,” in 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA) , Oct 2015, pp. 1–7
2015
Later among the works it cites.
B. Baingana and G. B. Giannakis, “Joint community and anomaly tracking in dynamic networks,” IEEE Transactions on Signal Processing , vol. 64, no. 8, pp. 2013–2025, April 2016
2016
Later among the works it cites.
J. Sharpnack, A. Rinaldo, and A. Singh, “Detecting anomalous activity on networks with the graph fourier scan statistic,” IEEE Transactions on Signal Processing , vol. 64, no. 2, pp. 364–379, Jan 2016
2016
Later among the works it cites.
S. Wisdom et al. , “Full-capacity unitary recurrent neural networks,” in Advances in Neural Information Processing Systems , 2016, pp. 4880–4888
2016
Later among the works it cites.
L. M. Candanedo and V. Feldheim, “Accurate occupancy detection of an office room from light, temperature, humidity and co2 measurements using statistical learning models,” Energy and Buildings , vol. 112, no. Supplement C, pp. 28 – 39, 2016
2016
Later among the works it cites.
S. Rayana, “ODDS library,” 2016. [Online]. Available: http://odds.cs.stonybrook.edu
2016
Later among the works it cites.
T. Xie, N. M. Nasrabadi, and A. O. Hero, “Learning to classify with possible sensor failures,” IEEE Transactions on Signal Processing , vol. 65, no. 4, pp. 836–849, Feb 2017
2017
Closest in time.
K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber, “LSTM: A search space odyssey,” IEEE Transactions on Neural Networks and Learning Systems , vol. 28, no. 10, pp. 2222–2232, Oct 2017
2017
Closest in time.
L. Jing, C. Gulcehre, J. Peurifoy, Y. Shen, M. Tegmark, M. Soljačić, and Y. Bengio, “Gated orthogonal recurrent units: On learning to forget,” arXiv preprint arXiv:1706.02761 , 2017
Original
2017
Closest in time.
M. Bai, B. Zhang, and J. Gao, “Tensorial recurrent neural networks for longitudinal data analysis,” arXiv preprint arXiv:1708.00185 , 2017
Original
2017
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…