Real-Time Tracking Combined with Object Segmentation

Abstract

We propose a new approach that integrates object tracking with object segmentation in a closed loop. The EM-like algorithm for color-histogram-based object tracking is modified to deal with the appearance models of the object and background represented by the Gaussian mixture models which are more efficient in RGB color space. It provides a rough object spatial model to guide segmentation. A five-layer region based graph cuts algorithm is developed to extract the accurate object region based on the object spatial model. It is effective even in cluttered background and runs more than 10 times as fast as Grab Cut. Then we can establish the appearance models of the object and background avoiding introducing errors and update them frame by frame without the problem of drift. The refined and adaptive models lead to robust tracking in return. Moreover, the motion of the object is estimated to produce a predicted object location in the new frame for tracking. A real-time robust tracking system is built based on the proposed approach and validated on a variety of challenging sequences.