Multibranch Adaptive Fusion Network for RGBT Tracking

Abstract

RGBT tracking has been increasingly investigated in visual tracking due to the strong complementary nature of visible and infrared images. However, in the established RGBT tracking algorithms, multiscale information has not been well exploited and utilized, which limits the performance of the tracker. In this paper, a novel multibranch adaptive fusion network is proposed, which aggregates multiscale information from multiple branches. Specifically, our backbone network draws on the modified VGG-M. To extract the multiscale features, we design a multiscale adapter, which adds two small convolution kernel branches to the backbone in each layer and each modality in a parallel manner. We also design a multibranch fusion module to adaptively aggregate the features from multiple branches and the previous layer. Moreover, we propose a multimodal fusion module for aggregating features between modalities, which could mitigate the impact of noise from low-quality sources. Finally, many results on two recent RGBT tracking datasets show that our method significantly outperforms other state-of-the-art tracking methods.