Cascaded Alternating Refinement Transformer for Few-shot Medical Image Segmentation

Abstract

Conventional biomedical image segmentation heavily relies on substantial annotations, which demand significant human and financial resources for collection. Consequently, learning a model with excellent performance using limited medical image data becomes a challenging problem. Upliftingly, the advent of few-shot medical image segmentation (FSMIS) offers a potential solution. Although prototypical networks are commonly employed in existing FSMIS tasks, the prototypes derived from support features often induce significant bias issues caused by intra-class variations. To this end, we propose a method called Cascaded Altering Refinement Transformer (CART) to iteratively calibrate the prototypes with both support and query features. This method focuses on capturing the commonality between foreground information of the support and query features using the Alterating Refinement Transformer (ART) module, which includes two Multi-head Cross Attention (MCA) modules. Furthermore, we cascade ART modules to refine the class prototypes, resulting in representative prototypes. This process ultimately contributes to a more accurate predicted mask. Besides, to preserve more valid information in each cascaded ART module and achieve better performance, we propose a novel inference method that accumulates the predicted segmentation masks in all ART modules by applying the Rounding-Up strategy. Extensive experiments on three public medical image datasets demonstrate that our model outperforms the state-of-the-art methods, and detailed analysis also validates the reasonableness of this design. Code is available at: https://github.com/zmcheng9/CART .