A Simple Masked Autoencoder Paradigm for Point Cloud

Abstract

Unsupervised pre-training is a promising approach to address the problem of laborious manual annotation, which has attracted great attention in 3D point clouds. Recent works focus on corruption-reconstruction methods that corrupt the input data first and then learn to reconstruct the uncorrupted data, but they still lack simplicity and generality. To solve this problem, we have simplified traditional unsupervised methods for point clouds. We propose MPE, a paradigm that is based on group Masked for Point cloud autoEncoder, which is simple to be implemented and can be applied to various model architectures. Specifically, 1) MPE adopts a random group mask to corrupt the input cloud data for reconstruction learning. 2) Various model architectures, like CNN, Edge- Conv, Attention, or the hybrid of them, can be pre-trained under this strategy. 3) A lightweight prediction head acts as a decoder and performs better than heavier ones. The pre-trained models can be used as a great initialization for different downstream tasks like classification and segmentation. Extensive experiments demonstrate that the proposed method can effectively improve the performance of various models. Code is available at https://github.com/zixiangro/MPE.