MVPointNet: Multi-View Network for 3D Object Based on Point Cloud

Abstract

The research of hand pose estimation is a hot topic in computer vision, robotics and virtual reality. Compared with using data glove, vision based methods show great advantage for its contactless property, low-cost and convenience. With the commercial depth cameras became widely available and the great success of Convolution Neural Network (CNN) on images, various works focused on hand pose estimation have achieved promising performance. This research is inspired by the recent work that directly perform 3D classification and segmentation tasks on point cloud. In this paper, Multi-View PointNet (MVPointNet) is proposed which takes several views of point cloud as input source. Then, they fed into the well-performing point cloud-based architecture. In addition, to better capture the hand context structure and improve the performance, more features between centroid and local neighborhood points (norm, edge, angle) are extracted and fed into a deep CNN architecture. To our knowledge, our proposed method achieved good performance on the ModelNet40 dataset for 3D shape classification. Besides, it achieved superior performance over other deep learning methods for 3D hand pose estimation based on point cloud, which is evaluated on MSRA dataset.