TBRAFusion: Infrared and visible image fusion based on two-branch residual attention Transformer

Abstract

<p>The fusion of infrared and visible images highlights the target while preserving detailed information, which helps to comprehensively capture the scene information. However, the existing methods continue to face challenges in managing the integration of global and local information, as well as enhancing the extraction of detailed image features, thus ultimately leading to constrained fusion outcomes. To enhance the fusion effect, this paper proposes a dual-branch residual attention-based infrared and visible image fusion network (TBRAFusion). The network utilizes two key modules, TransNext and the dual-branch residual attention (DBRA) module, which are used to process the input images in parallel to extract contrast and detail information. Additionally, an auxiliary function is incorporated into the loss function. Compared with mainstream fusion models, TBRAFusion achieves better fusion results and metrics through these improvements. The experimental results on the TNO dataset show that TBRAFusion improves the metrics in entropy (EN), spatial frequency (SF), sum ofcorrelation differences (SCD), and visual information fidelity (VIF) by 0.42$ \% $, 4$ \% $, 3.9$ \% $, and 1.2$ \% $, respectively. Tests on the MSRDS dataset show improvements of 1.7$ \% $, 5.4$ \% $, 9.6$ \% $, and 4.9$ \% $ in EN, standard deviation (SD), SF, and SCD, respectively.</p>