SAMNet++: A Segment Anything Model for Supervised 3D Point Cloud Semantic Segmentation

Abstract

Segmentation of 3D point clouds is essential for applications such as environmental monitoring and autonomous navigation, where making accurate distinctions between different classes from high-resolution 3D datasets is critical. Segmenting 3D point clouds often requires a trade-off between preserving spatial information and achieving computational efficiency. In this paper, we present SAMNet++, a hybrid 3D segmentation model that integrates segment anything model (SAM) and adopted PointNet++ in a sequential two-stage pipeline. Firstly, SAM performs an initial unsupervised segmentation, which is then refined using adopted PointNet++ to improve the accuracy. The key innovations of SAMNet++ include its hybrid architecture, which combines SAM’s generalization with PointNet++’s local feature extraction, and a feature refinement strategy that enhances precision while reducing computational overhead. Additionally, SAMNet++ minimizes the reliance on extensive supervised training, while maintaining high accuracy. The proposed model is tested on three urban datasets, which are collected by an unmanned aerial vehicle (UAV). The proposed SAMNet++ model demonstrates high segmentation performance, achieving accuracy, precision, recall, and F1-score values above 0.97 across all classes on our experimental datasets. Furthermore, its mean intersection over union (mIoU) of 86.93% on a public benchmark dataset signifies a more balanced and precise segmentation across all classes, surpassing previous state-of-the-art methods. In addition to its improved accuracy, SAMNet++ showcases remarkable computational efficiency, requiring almost half the processing time of standard PointNet++ and nearly one-sixteenth of the time needed by the original PointNet algorithm.