Siammraan: Siamese Multi-Level Residual Attention Adaptive Network for Hyperspectral Videos Tracking

Abstract

The deep learning based techniques have been widely applied to object tracking in color videos. When these techniques are applied to hyperspectral videos, how to fully explore unique spectral signatures of tracking objects is of crucial importance as well as simultaneously utilizing spatial and temporal information. Different with color videos, hyperspectral videos record continuous spectral reflectance of targets in light wavelength indexed band images and it is more difficult to explore unique spectral feature of tracking objects. Aiming to take advantage of existing object tracking techniques in color videos, a Siamese Multi-level Residual Attention Adaptive Network (SiamMRAAN) is designed to handle 3-band images by using the well-trained ResNet50 as backbone. By grouping hyperspectral videos into several 3-band-image subsets, the proposed SiamMRAAN can be used to explore high-dimensional spectral information. We design a loss function to fuse the tracking results over these subsets to improve the tracking performance. Finally, experiments over 75 hyperspectral videos confirmed that using spectral information is critical to improve the performance of object tracking in color videos, and also demonstrated that the proposed SiamMRAAN based strategy outperforms several compared networks for hyperspectral videos.