Multiscale Multilevel Residual Feature Fusion for Real-Time Infrared Small Target Detection

Abstract

Detecting infrared dim and small targets is one crucial step for many tasks such as early warning. It remains a continuing challenge since characteristics of infrared small targets, usually represented by only a few pixels, are generally not salient. Despite that many traditional methods have significantly advanced the community, their robustness or efficiency is still lacking. Most recently, CNN-based object detection has achieved remarkable performance and some researchers focus on it. However, these methods are not computationally efficient when implemented on some CPU-only machines and few datasets are available publicly. To promote the detection of infrared small targets in complex backgrounds, we propose a new lightweight CNN-based architecture. The network contains three modules: the feature extraction module is designed for representing multi-scale and multi-level features, the grid resample operation module is proposed to fuse features from all scales, and a decoupled head to distinguish infrared small targets from backgrounds. Moreover, we collect a brand-new infrared small target detection dedicated dataset which consists of 68311 practical captured images with complex backgrounds for alleviating the data dilemma. To validate the proposed model, 54758 images are used for training and 13553 images are used for testing respectively. Extensive experimental results demonstrate that the proposed method outperforms all traditional methods by a large margin and runs much faster than other CNN methods with high precision. The proposed model can be implemented on the Intel i7-10850H CPU (2.3GHz) platform and Jetson Nano for real-time infrared small target detection at 44 FPS and 27 FPS, respectively. It can be even deployed on an Atom x5-Z8500 (1.44GHz) machine at about 25 FPS with 128×128 local images. The source codes and the dataset have been made publicly available at https://github.com/SeaHifly/Infrared-Small-Target.