Semi-supervised Anomaly Detection with Reinforcement Learning

Abstract

Reconstruction-based anomaly detections with convolutional autoencoders (CAEs) have been commonly used for unsupervised anomaly detection. The task of anomaly classification and segmentation is carried out by calculating the error between the reconstructed output and its original input with a pre-determined threshold. However, the process of determining a suitable threshold is a timely process that requires an extra search process with a pre-defined minimum defect area that must be optimized for each type in various classes. Consequently, the resulting detection performance becomes highly biased on the threshold. Therefore, the underlying principle of using a fixed threshold value for a per-pixel anomalous region decision is questionable. To address this issue, we propose a deep reinforcement learning approach to learn an optimal policy that can differentiate between anomalous and normal samples from the given residual map. Empirical experiments on the MVTec anomaly detection dataset demonstrate that the proposed method significantly improves detection performance without changing the residual map and can be further enhanced depending on the input to the policy network model.