Improving intrusion detection for imbalanced network traffic

Abstract

ABSTRACT The acceptability and usability of intrusion detection systems get seriously affected with data imbalance in network traffic. A large number of false alarms mean a lot in terms of the acceptability of intrusion detection systems. The reason for the increase in false alerts is that the normal traffic abound. Even with highly accurate intrusion detection systems, the effective detection rate of the minority attack types will be unacceptably low, and those attack types are often the most serious ones. Thus, high accuracy is not necessarily an indicator of high model quality, and therein lays the accuracy paradox of predictive analytics. The cost of missing an attack is higher than the cost of false alarms. The aim of this work is to provide an architecture that enables available intrusion detection systems to work together towards creating a more realistic model of the state of a network. The data‐dependent decision fusion architecture presented in this paper learns from the data and then appropriately gives weighting to the decisions of various intrusion detection systems. The fusion enriches these weighted decisions to provide a single decision, which is better than those of the existing intrusion detection systems. It is also shown that our technique is more flexible and also outperforms other existing fusion techniques such as OR, AND, SVM, and ANN. This method reduces the false positive rate and improves the overall detection rate and, also, the detection rate of minority class types in particular. For illustrative purposes, two different data sets, namely the DARPA 1999 data set as well as the real‐time network traffic embedded with attacks, have been used. Copyright © 2012 John Wiley & Sons, Ltd.