PSCT-Net: A parallel symmetric CNN-transformer hybrid network for medical image segmentation

Abstract

The precision of medical image segmentation is important in clinical analysis and diagnosis. CNN-Transformer based hybrid approaches show great potential in medical image segmentation due to their complementarity in modeling local and global contextual dependencies. However, local representations and global representations possess their own distinct structures and semantic characteristics, simplistic or inappropriate fusion strategies are insufficient to leverage their complementary strengths, Impeding the model to achieve optimal segmentation performance. For resolving this dilemma, we proposed A Parallel Symmetric CNN-Transformer Hybrid Network for Medical Image Segmentation (PSCT-Net)that implements a three-phase fusion mechanism to sufficiently and efficiently fuse heterogeneous and complementary features: 1) During the encoding stage, we design a layer-wise feature fusion (LWFF) module efficiently merges both CNN and Transformer learned local and global feature, enabling the network to learn more distinctive multi-scale feature. 2) For skip connections, we introduce a multi-scale feature fusion (MSFF) module to capture spatial and channel dependencies among features from different encoding layers while filtering redundant information through multi-scale feature spatial fusion (MFSF) and multi-scale feature channel fusion (MFCF). 3) In the decode stage, We also adopt a dual-branch architecture and through the LWFF module integrates upsampled features from the same decode layer enables the network to more accurately restore the image resolution information. Additionally, we through the CrossTransformer block further enhance the network's capability in processing boundary details. Comprehensive experiments on four medical datasets demonstrate the superiority, effectiveness, and robustness of our PSCT-Net.