Cross Attention with Transformer for Few-shot Medical Image Segmentation

Abstract

Few-shot medical segmentation aims at learning to segment a new organ object using only a few annotated support images of the target class, which require the network to have strong generalization performance in extremely few data regimes. There exists a groundbreaking idea of integrating meta-learning methods into few-shot medical image segmentation. However, existing few-shot medical image segmentation methods fail to consider the global anatomy correlation between the support and query set. The information exchange between two branches is too weak to fully carry out a sufficient semantic understanding. This determines how much knowledge the query set learns from the corresponding support set. To alleviate the above-mentioned drawbacks, we propose a novel Cross Attention network based on traditional two-branch methods. Our starting point is to fully integrate the global information of the two parts with the frequently used fusion method in cross-modality tasks. To achieve this goal, we raised two main contributions: (1) The transformer-based Cross Attention module is leveraged to strengthen the fusion of support and query data. and (2) Through the comparable results on two challenging datasets (abdominal segmentation dataset CHAOS and cardiac segmentation dataset MS-CMRSeg), we proved that the traditional meta-learning based methods still have great potential when strengthening the information exchange between two branches. We achieve remarkable improvement in the proposed method compared with currently dominant metric learning-based methods.