Visual tracking of hand posture with occlusion handling

Abstract

We propose a real-time visual hand tracking and posture estimation system to study prehension gestures. This system is based on an original 26 degree-of-freedom model of the hand, for which forward and inverse kinematics formulations have been developed. The hand, wearing a dark glove marked with colored cues, is tracked in real time in the image. The position of the color markers in the image is used to reconstruct the pose of the hand and the postures of the fingers. Occlusion handling is achieved by prediction of finger positions and is validated by testing 3D geometric visibility conditions. Results on real and synthesized image sequences with finger occlusions are presented.