SoccerSAM: Leveraging Segment Anything Model for Unlabeled Semantic Segmentation of Football Match Scenes

Abstract

Introduction: This paper addresses a novel and challenging task: how to leverage the emerging Segment Anything Model (SAM), which demonstrates impressive zero-shot instance segmentation capabilities, to train a compact semantic segmentation model for football scenes (student) without requiring any labeled data. Objectives: This presents significant challenges due to SAM’s inability to provide semantic labels and the considerable capacity gap between SAM and the student model. To solve this, we introduce a novel framework, SoccerSAM, which incorporates a Semantic Bridging Module (SBM) to bridge this gap. Methods: The SBM assigns semantic class probabilities to SAM-generated instance masks, producing dense semantic logits for training. To further enhance the model’s boundary accuracy, we introduce a Boundary-Aware Consistency Loss that aligns the predicted edges with SAM’s high-quality boundary information. Additionally, we propose a Logit-Level Consistency Loss to enforce alignment between the student’s predictions and the pseudo-labels generated by SBM. Results:Extensive experiments on the FSSOD dataset show that our SoccerSAM outperforms previous methods, achieving significant improvements in both mIoU and FWIoU, with a remarkable boost in performance, especially in small object segmentation, while maintaining a lightweight architecture. Conclusions: In this paper, we introduced SoccerSAM, a novel framework for football scene segmentation that leverages the Segment Anything Model (SAM) and the Semantic Bridging Module (SBM) to generate soft semantic supervision without requiring labeled data. Extensive experiments on the FSSOD dataset demonstrate that SoccerSAM outperforms existing methods in terms of both mIoU and FWIoU, achieving significant improvements, especially in small object segmentation. Our approach provides a lightweight and effective solution for real-time football scene segmentation, and future work will explore extending the framework to handle more complex scenes and incorporate additional domain-specific knowledge.