Knowledge Distillation for Anomaly Detection

Abstract

Unsupervised deep learning techniques are widely used to identify anomalous behaviour.The performance of such methods is a product of the amount of training data and the model size.However, the size is often a limiting factor for the deployment on resource-constrained devices.We present a novel procedure based on knowledge distillation for compressing an unsupervised anomaly detection model into a supervised deployable one and we suggest a set of techniques to improve the detection sensitivity.Compressed models perform comparably to their larger counterparts while significantly reducing the size and memory footprint.