Anomaly Detection in Data with Extremely High Dimensional Space via Online Oversampling Principal Component Analysis

Abstract

Anomaly detection is a crucial analysis topic in the field of data mining as well as machine learning.Several real-world applications like Intrusion or MasterCard fraud detection need a good and efficient framework to spot deviated data instances.A good anomaly detection methodology must be able to accurately establish many varieties of anomalies, robust, need comparatively very little resources, and perform detection in period of time.In this paper we proposed the idea of combining the two different algorithms i.e.Median Based Outlier Detection and Online Oversampling PCA for effective detection of anomaly in online updating mode.Median Based outlier detection uses the interquartile range which is a measure of statistical dispersion being equal to the difference between the upper and lower quartiles.Whereas oversampling PCA does not need to store the entire covariance matrix or data matrix and thus this approach is a more useful in online or large scale problem.Compared with other anomaly detection algorithm our experimental result verifies the feasibility of our proposed method.