Quantity-Quality Enhanced Self-Training Network for Weakly Supervised Point Cloud Semantic Segmentation

Abstract

Point cloud semantic segmentation is essential for understanding 3D scenes. Contemporary techniques often require extensive annotated training data, yet obtaining point-wise annotations for point clouds is time-consuming and laborious. Recent developments in weakly supervised methods seek to mitigate this problem by generating pseudo-labels using limited annotations. However, these pseudo-labels frequently suffer from either insufficient quantity or inferior quality. To overcome these hurdles, we introduce a Quantity-Quality Enhanced Self-training Network for Weakly Supervised Point Cloud Semantic Segmentation (Q2E). Specifically, an image-assisted pseudo-label generator is proposed to exploit 2D images to extend pseudo-labels for point clouds. Additionally, a hierarchical pseudo-label optimizer is developed to refine the quality of the pseudo-labels by hierarchically grouping them into broader categories. Extensive experiments on the ScanNet-v2, S3DIS, Semantic3D, and SemanticKITTI datasets demonstrate that Q2E outperforms state-of-the-art weakly supervised methods and rivals fully supervised approaches for point cloud semantic segmentation. Remarkably, as of the initial submission on February 2, 2024, our method ranked the first place in various settings of the ScanNet-v2 benchmark.