Hybrid Method for Network Anomaly Detection: Combining Clustering and Classification with Feature Selection

Abstract

In this study, we have devised an innovative method for automating the classification of network attacks, leveraging a hybrid approach to feature selection. By employing this technique, we were able to dynamically reduce the feature set from an initial 47 to a more manageable 15, streamlining the analysis process while retaining crucial information for the accurate identification of attack patterns. The method also integrates a clustering classification framework, where K-means clustering serves as the foundation for grouping similar data points. To determine the optimal number of clusters, we employed the elbow method, resulting in the selection of a value for k that maximizes cluster homogeneity. Through rigorous validation, we ensured the robustness of our clustering approach, achieving a silhouette coefficient of 0.7048, indicative of well-defined and distinct clusters. Subsequently, we trained and evaluated an XGBOOST algorithm on our refined dataset. The XGBOOST algorithm, renowned for its effectiveness in handling structured data and classification tasks, exhibited remarkable performance. Our model achieved an outstanding overall accuracy rate of 0.9991, underscoring its proficiency in accurately classifying network attacks with a high degree of precision and reliability.