KD-LightMAD: Knowledge Distillation-Based Lightweight Multimodal Anomaly Detection Framework for Smart Manufacturing

Abstract

With the advancements in smart manufacturing environments, the importance of anomaly detection methods that integrate and analyze heterogeneous multisensor data, e.g., partial discharge images and time-series signals, is becoming increasingly prominent. However, existing high-performance multimodal models are limited in terms of their deployment in real-time systems or on edge devices due to their large number of parameters and high computational requirements. To address these issues, this paper proposes the knowledge distillation-based lightweight multimodal anomaly detection (KD-LightMAD) framework. The proposed framework is lightweight and achieves sufficient efficiency without compromising performance by inheriting the core feature information from MAD, a teacher model that combines RealNVP -based normalization flow, LIMoE-based expert selection structure, and SupCon-based contrastive learning. Experimental results demonstrate that the proposed KD-LightMAD framework achieves an ultralightweight size of only 15 MB by reducing the number of parameters by more than 98% compared with the teacher model, and it obtained an F1-score of 100.0%, thereby achieving performance that is equal to or better than that of existing state-of-the-art (SOTA) models. For example, the proposed framework realizes exceptional efficiency by reproducing the same performance as a 263-MB SOTA model at approximately 6% of its size. The findings of this study demonstrate that the proposed KD-LightMAD framework effectively fuses high-dimensional complex sensor data while maintaining real-time performance and accuracy, thereby enhancing the practicality and scalability of edge device-based anomaly detection systems for smart manufacturing.