TGAP-Net: Twin Graph Attention Pseudo-Label Generation for Weakly Supervised Semantic Segmentation

Abstract

Multilabel pathological tissue segmentation is a vital task in computational pathology that aims to semantically segment different tissues within pathological images. Fully and weakly supervised models have demonstrated impressive performances in this regard. However, weakly supervised models still face challenges, such as the poor performance of nondominant samples and limited effectiveness of aggregation functions in conveying supervisory signals. To address these issues, we propose two key contributions: the introduction of a graph attention network(GAT) module to establish contextual relationships between pixels within patches and generate high-quality pseudo-labels, and the development of a novel global classified max pooling(GCMP) aggregation function that effectively transmits the supervision signal from weakly annotated labels and improves the model's classification accuracy. The experimental results show that our method improved the MIoU scores by 3.3 and 3 for the nondominant samples, necrosis(NEC) and lymphocytes(LYM), respectively, in the LUAD-HistoSeg test set. This led to an overall MIoU of 0.774, which is a 1.8 increase in the state-of-the-art(SOTA) performance. Similarly, our approach improved MIoU scores by 5.7 and 2 on the NEC and LYM samples, respectively, in the Breast Cancer Semantic Segmentation(BCSS) test set, resulting in an overall MIoU of 0.721. This represents a 1.6 increase in SOTA performance. In summary, our work addresses the issues of poor performance on nondominant samples and the suboptimal performance of aggregation functions. We propose a novel approach to achieve a significant performance improvement. This is extremely significant in reducing the workload of manual annotation and promoting the development of computational pathologies.