Network traffic analysis based on collective anomaly detection

Abstract

There is a growing interest in the data mining and network management communities to improve the existing techniques for prompt analysis of underlying traffic patterns. Anomaly detection is one such technique to detect abnormalities in many different domains including computer network intrusion, gene expression analysis, financial fraud detection and many more. In this paper, we develop a framework to discover interesting traffic flows, which seem legitimate but are targeted to disrupt normal computing environment, such as Denial of Service attack. We propose a framework for collective anomaly detection using x-means clustering, which is a variant of basic k-means algorithm. We validate our approach by comparing against existing techniques and benchmark performance. Our experimental results are based on widely accepted DARPA dataset for intrusion detection from MIT Lincoln Laboratory.