Ensemble of Instance Segmentation Models for Polyp Segmentation in Colonoscopy Images

Abstract

Colorectal cancer is the second most frequently diagnosed cancer in women and the third most frequently diagnosed cancer in men. At least 80%-95% of the colorectal cancers are evolved from intestinal polyps. Although colonoscopy is regarded as the most effective method for screening and diagnosis, the success of the procedure is highly dependent on the level of hand-eye coordination and the operator skills. Thus, we are primarily motivated by the need for obtaining an early and accurate diagnosis of polyps in the colonoscopy images. In this paper, we employed the powerful object detection neural network “Mask R-CNN” to identify and segment polyps in the colonoscopy images. Also, we proposed an ensemble method to combine the two Mask R-CNN models with different backbone structures (ResNet50 and ResNet101) to enhance the performance. Mask R-CNNs in our model were first trained on COCO dataset, and then finely tuned using intestinal polyp dataset since a large number of annotated colonoscopy images are not easily accessible. In order to evaluate our proposed model, we used three open intestinal polyp datasets, CVC-ClinicDB, ETIS-Larib, and CVC-ColonDB. Our results show that our transfer learning-based ensemble model significantly outperforms state-of-the-art methods.