Pyramid Co-Attention Compare Network for Few-Shot Segmentation

Abstract

Few-shot segmentation (FSS), which aims to extract never learned classes of objects from query images with a few annotated support samples, is a challenging problem especially in the cases that the appearance of objects in the support and the query images is significant different. Therefore, we propose a deep network called Pyramid Co-Attention Compare Network (PCCNet) to narrow the gap between them by introducing a Pyramid Co-attention Module (PCAM). PCAM acts as a task-specific transformer to transform the features of corresponding objects in query and support images into a space in which they are much closer by taking advantage of the underlying relation between query and support images. We also introduce a Prototypical Guide Module (PGM) which uses non-parametric metric learning to guide parametric metric learning so as to combine the advantages of them. In addition, a Superpixel Refine Module(SRM) is proposed to optimize the final output segmentation masks. Experiments conducted on Pascal- $5^{i}$ shows that our PCCNet achieves a mean Intersection-over-Union(mIoU) score of 63.01% for 1-shot segmentation and 64.57% for 5-shot segmentation, outperforming state-of-the-art methods by margin of 2.2% and 1.6%, respectively.