Enhancing Intrusion Detection Systems with Ensemble Models and Hybrid Feature Selection Techniques

Abstract

Detection Systems (IDS) play a critical role in safeguarding networks against cyber attacks. However, selecting the most effective machine learning model for intrusion detection is challenging due to varying dataset characteristics. This research investigates the performance of multiple machine learning models, including SVM (Linear, Poly, RBF, and Sigmoid), LightGBM, XGBoost, and CatBoost, across two widely used datasets: CICIDS2017 and NF-UNSW-NB15. The primary problem is the inconsistency in model performance across different datasets, affecting the reliability of IDS solutions. To address this, we used SMOTE for balancing class distributions and PCA for dimensionality reduction. Each model was evaluated based on accuracy, precision, sensitivity, and F-measure. The results show that LightGBM, XGBoost, and CatBoost consistently outperform the SVM models across both datasets, with accuracy levels above 98%. In contrast, the SVM models exhibited significant variation, performing better on the NF-UNSW-NB15 dataset than on CICIDS2017. Ensemble models are more suitable for intrusion detection due to their higher and more stable performance across different datasets, making them preferable for real-world applications.