Dual-Teacher Guided Denoising Distillation for Anomaly Detection

Abstract

Recently, knowledge distillation-based approaches have demonstrated efficacy in unsupervised anomaly detection. These methods typically compute pointwise feature discrepancies between teacher and student (T–S) to localize anomalies. However, this paradigm suffers from two issues that degrade performance: 1) some normal regions exhibit large feature differences between T-S networks (leading to false detections), while 2) certain anomalous regions produce small differences (causing missed detections). To address this, we propose a Dual-Teacher Guided Denoising (DTGD) distillation framework that reformulates anomaly detection as a noise-removal task, where anomalies are treated as noise. Specifically, the DTGD framework includes a normal teacher, an anomaly teacher, and a denoising encoder-decoder student. The normal teacher encodes pristine normal data, and the anomaly teacher captures potential anomaly features from synthetic anomalies, guiding the student network to retain normal features and expel noise. After each encoder block of the student, our teacher-guided noise removal (TNR) module injects knowledge from the anomaly teacher, explicitly teaching the student which features to preserve. Furthermore, consistency loss and dissimilarity loss functions enforce consistency with the normal teacher in both encoder/decoder outputs and increase dissimilarity of anomaly-related features from the anomaly teacher via pixel-wise supervision. Experiments on anomaly detection datasets demonstrate that DTGD achieves advanced localization accuracy and produces sharper anomaly maps, reducing both false positives and false negatives.