MKTformer: Fine-grained Meter Classification Based on Multi-modal Knowledge Transfer

Abstract

Meter classification in converter station lays the foundation for the subsequent detection-related tasks, however, training excellent meter classification models takes a large quantity of data and computing resources. Aiming to solve this problem, a fine-grained meter classification method based on multi-modal knowledge transfer is proposed. Firstly, the multi-modal knowledge of the Contrastive Language-Image Pre-training (CLIP) model is transferred to provide a more general visual representation for fine-grained features extraction. Secondly, a Task Space Mapping Unit (TSMU) is designed to improve the transfer ability of the multi-modal knowledge. Finally, a new transfer learning strategy is proposed on this basis to achieve a better transfer performance. The experimental results show that our method can achieve higher accuracy than its counterpart in significantly less train time under both fully supervised and few-shot settings, which verifies the its superiority in capturing fine-grained features and reducing training cost.