Depth Swin Transformer Unet for Serial Section Biomedical Image Segmentation

Abstract

Accurate segmentation of three-dimensional serial-section electron microscopy(EM) images is of great significance for neuroscience research. For anisotropic 3D EM image segmentation, it is necessary to obtain information in all dimensions in a balanced manner. However, many existing methods default that images are isotropic, leading to difficulty in obtaining high quality results in anisotropic EM images. In this work, we proposed Depth Swin Transformer Unet(DSTUnet), aU-shaped neural network based on Depth Swin Transformer (DST) block and convolution layers, to improve the segmentation of 3D EM images. DST block is composed of a Swin Transformer block with a flattened window to get a larger receptive field in the X-Y direction and a Depth Overlapped Attention (DOA) module to enable the transmission of neighborhood pixel information in depth-wise. The proposed DOA module employed a slightly larger and overlapped patch in the key and value vector, which significantly promotes the model performance. Taking advantage of DSTUnet, we realized segmentation of mitochondria in the MitoEM-R, Lucchi++ dataset, and neural cell from optical images, achieving superior performance to existing methods.