Segment Anything Model (SAM) Meets Object Detected Box Prompts

Abstract

Segmenting images is an intricate and exceptionally demanding field within computer vision. Instance Segmentation is one of the subfields of image segmentation that segments objects on a given image or video. It categorizes the class labels according to individual instances, ensuring that distinct instance markers are assigned to each occurrence of the same object class, even if multiple instances exist. With the development of computer systems, segmentation studies have increased very rapidly. One of the state-of-the-art algorithms recently published by Meta AI, which segments everything on a given image, is called the Segment Anything Model (SAM). Its impressive zero-shot performance encourages us to use it for diverse tasks. Therefore, we would like to leverage the SAM for an effective instance segmentation model. Accordingly, in this paper, we propose a hybrid instance segmentation method in which Object Detection algorithms extract bounding boxes of detected objects and load SAM to produce segmentation, called Box Prompted SAM (BP-SAM). Experimental evaluation of the COCO2017 Validation dataset provided us with promising performance.