SwinResNet: Volumetric Medical Image Segmentation by Fusing Swin Transformer and ResNet

Abstract

Volumetric medical image segmentation is critical in diagnosing diseases and planning subsequent treatment. The convolutional neural network (CNN)-based U-Net was proposed for conducting accurate and robust medical image segmentation since the skip connection of U-Net and deep feature representation significantly improved its performance. However, since CNN-based models mainly focus on local and low-level features, they cannot extract global and high-level features effectively. Meanwhile, the Vision Transformer developed in natural language processing is proposed to improve image classification performance by splitting an input image into patches and conducting linear embeddings of the patches, which can extract global features. However, the Vision Transformer has difficulty in handling detailed and low-level features. This study proposes SwinResNet which can effectively conduct volumetric medical image segmentation by fusing the Swin Transformer and CNN models. The combination can take advantage of both models and complement each other. Swin Transformer and ResNet are used as encoders, and the receptive field blocks and aggregation modules are applied to the multi-level features extracted from both encoders. Comprehensive evaluation shows that the proposed approach outperforms well-known previous studies.