Graph Convolutional Neural Networks-based 3D Hand Pose Estimation over Point Clouds

Abstract

In recent years we can find a multitude of approaches that aim to return the 3D pose of the hands. Most of them try to estimate the pose from RGB images or even include some geometrical information via depth maps. Furthermore, some proposals have shown promising results using point clouds as input data. However, the sparse nature of this type of data is often one of its drawbacks. To tackle this sparsity, different strategies have been brought to the table such as voxelizing or sorting the input data to impose a structure to the input domain. In this paper, we address this problem by means of a graph structure. This process implies that we should accommodate the point cloud onto a graph representation that connects its points. We connect each point to its neighborhood, a method that has been successfully used in similar proposals and whose clustering effect enables us to emulate an effect similar to kernels in image convolutions. The proposed architecture uses both graph and 2D convolutions. The first one aims to extract local features and build a feature map, from which the 2D convolutions will extract a second level of features used to estimate the pose. This proposal shows initial results to return a 3D pose of the hand from depth maps, which are projected on point clouds and redefined as graphs. Although the results diverge from other more established methods in the state of the art, it presents a proof of concept by which to address this problem without losing spatial information.