A Multi-scale Fusion Network with Transformer for Medical Image Segmentation

Abstract

To further exploit the advantages of convolutional neural network (CNN) and Transformer, we introduce a new Multi-scale Fusion Network to this paper. With the U-shaped attention model, we introduce multi-scale blocks in the encoder phase to sufficiently exploit the multi-scale semantic information. We further invoke cross-fusion of the multi-scale channels with Transformer to reconstruct skip connections, which provides the decoder with different levels of long-range information. Moreover, we utilize a scale-aware pyramid fusion module built into the bottom of our framework for the dynamic fusion of multiscale contextual information from higher-level features. The results on two datasets indicate that the proposed approach obtains competitive performance and exceeds the comparison networks, which to some extent relieves the burden of physicians.