Learning Sparse Representation With Variational Auto-Encoder for Anomaly Detection

Abstract

Anomaly detection has a wide range of applications in security area such as network monitoring and smart city/campus construction. It has become an active research issue of great concern in recent years. However, most algorithms of the existing studies are powerless for large-scale and high-dimensional data, and the intermediate data extracted by some methods that can handle high-dimensional data will consume lots of storage space. In this paper, we propose a novel sparse representation framework that learns dictionaries based on the latent space of variational auto-encoder. For large-scale data sets, it can play the role of dimensionality reduction to obtain hidden information, and extract more high-level features than hand-crafted features. At the same time, for the storage of normal information, the space cost can be greatly reduced. To verify the versatility and performance of the proposed learning algorithm, we have experimented on different types of anomaly detection tasks, including KDD-CUP data set for network intrusion detection, Mnist data set for image anomaly detection, and UCSD pedestrian's data set for abnormal event detection in surveillance videos. The experimental results demonstrate that the proposed algorithm outperforms competing algorithms in all kinds of anomaly detection tasks.