Medical Image Fusion Using Unified Image Fusion Convolutional Neural Network

Abstract

Medical image fusion (IF) is a process of registering and combining numerous images from multiple‐ or single‐imaging modalities to enhance image quality and lessen randomness as well as redundancy for increasing the clinical applicability of the medical images to diagnose and evaluate clinical issues. The information that is acquired additionally from fused images can be effectively employed for highly accurate positioning of abnormality. Since diverse kinds of images produce various information, IF becomes more complicated for conventional methods to generate fused images. Here, a unified image fusion convolutional neural network (UIFCNN) is designed for IF utilizing medical images. To execute the IF process, two input images, namely, native T1 and T2 fluid‐attenuated inversion recovery (T2‐FLAIR) are taken from a dataset. An input image‐T1 is preprocessed employing bilateral filter (BF), and it is segmented by a recurrent prototypical network (RP‐Net) to obtain segmented output‐1. Simultaneously, input image‐T2‐FLAIR is also preprocessed by BF and then segmented using RP‐Net to acquire segmented output‐2. The two segmented outputs are fused utilizing the UIFCNN that is introduced by assimilating unified and unsupervised end‐to‐end IF network (U2Fusion) with IF framework based on the CNN (IFCNN). In addition, the UIFCNN obtained maximal Dice coefficient and Jaccard coefficient of 0.928 and 0.920 as well as minimal mean square error (MSE) of 0.221.