Robust 3D head tracking under partial occlusion

Abstract

This paper describes a novel system for 3D head tracking under partial occlusion from 2D monocular image sequences. In this system, the extended superquadric (ESQ) is used to generate a geometric 3D face model in order to reduce the shape ambiguity. Optical flow is then employed with this model to estimate the 3D rigid motion. To deal with occlusion, a new motion segmentation algorithm using motion residual error analysis is developed. The occluded areas are successfully detected and discarded as noise by the system. Also, accumulation error is heavily reduced by a new post-regularization process based on edge flow. This makes the system more stable over long occlusion image sequences. To show the accuracy, the system is applied on a synthetic occlusion sequence and comparisons with the ground truth are reported. To show the robustness, experiments on long occlusion image sequences, including synthetic and real ones, are reported.