Multi-camera hand pose recognition system using skeleton image

Abstract

We propose a vision-based hand pose recognition system. The system we propose expresses a hand pose by a plane model that consists of hand's center of gravity (COG) and fingertip points. These points of reference can be relatively more stable and easily detected than other points (e.g., finger base points). However, since it has been assumed in the COG detection process that a hand region in an image is separate from other regions, detection becomes unstable when, for instance, a hand region is connected with an arm region. Moreover, finger occlusion which occurs in specific ranges of palm direction, makes angle detection unstable. The technique we present here solves the former problem by using hand skeleton images detected by a multi-camera system. We picked many candidates as the COG and selected a candidate according to its attributes. The multi-camera system also solves the latter problem. Results of a series of experiments on the former problem are also presented.