A robust anomaly detection algorithm based on principal component analysis

Abstract

Quantifying the abnormal degree of each instance within data sets to detect outlying instances, is an issue in unsupervised anomaly detection research. In this paper, we propose a robust anomaly detection method based on principal component analysis (PCA). Traditional PCA-based detection algorithms commonly obtain a high false alarm for the outliers. The main reason is that ignores the difference of location and scale to each component of the outlier score, this leads to the cumulated outlier score deviates from the true values. To address the issue, we introduce the median and the Median Absolute Deviation (MAD) to rescale each outlier score that mapped onto the corresponding principal direction. And then, the true outlier scores of instances can be obtained as the sum of weighted squares of the rescaled scores. Also, the issue that the assignment of the weight for each outlier score will be solved. The main advantage of our new approach is easy to build with unsupervised data and the recognition performance is better than the classical PCA-based methods. We compare our method to the five different anomaly detection techniques, including two traditional PCA-based methods, in our experiment analysis. The experimental results show that the proposed method has a good performance for effectiveness, efficiency, and robustness.