Poison-Resilient Anomaly Detection: Mitigating Poisoning Attacks in Semi-Supervised Encrypted Traffic Anomaly Detection

Abstract

Semi-supervised encrypted traffic anomaly detection models in zero-positive scenarios are susceptible to human labeling errors or poisoning attacks, thereby compromising the stability and reliability of the model. However, existing methods are insufficient to address the challenge of reduced inter-class distance caused by poisoning attacks and the inability of reconstruction error to serve as a reliable detection criterion. To alleviate these challenges, a framework called Poison-Resistant Anomaly Detection (PRAD) is proposed to mitigate poisoning attacks and enhance anomaly detection performance. Specifically, a feature encoding module autoencoder-based is first designed that simultaneously leverages the Amsgrad gradient descent algorithm and the warm-up strategy to enhance the feature extraction and generalization capabilities, thereby alleviating the reduction of inter-class distance. Additionally, a feature analysis module is introduced to measure the impact of poisoning attacks on inter-class distance and the distribution of reconstruction errors, which provides valuable prior information for subsequent anomaly detection tasks. Finally, an online clustering-based anomaly detection algorithm that utilizes the extracted features and their corresponding reconstruction errors are developed to address the issue of detection criteria. Experimental results on public benchmark datasets demonstrate that PRAD exhibits significantly superior poison-resilient capabilities compared to other semi-supervised anomaly detection methods in anomaly detection tasks under poisoning attacks.