SFR: Semantic-Aware Feature Rendering of Point Cloud

Abstract

Multi-view projection methods have demonstrated their ability to reach state-of-the-art performance in point cloud downstream tasks(e.g., classification and retrieval). These methods first require rendering the point cloud into 2D multi-view images. However, conventional methods only project the geometry of the point cloud, and such projections inevitably suffer from a loss of point cloud semantic information due to dimensionality reduction. We propose a semantic-aware and task-oriented differentiable feature rendering (SFR), which reduces the information loss during projection by generating rendered images with more point cloud semantic information for downstream tasks. Our SFR method can be applied as a plug-and-play module added to any multi-view-based backbone network for end-to-end training. Extensive experiments on benchmark datasets show that our SFR method reaches state-of-the-art performance and brings general improvements to point cloud classification and retrieval tasks.