Video Object Extraction Based on Spatiotemporal Consistency Saliency Detection

Abstract

Video object extraction (VOE) refers to the challenging task of separating foreground objects automatically from a background. Aiming to resolve the problems of incomplete extraction of foreground objects and the background interference of irrelevant small moving objects, this paper proposes a new method of VOE based on spatiotemporal consistency saliency detection. The main innovation in this proposed method is composed of three parts: first, the spatiotemporal gradient field (SGF) is constructed by mutual consistency between the static gradient feature of intra-frame and the dynamic gradient feature inter-frame, and a coarse motion saliency map is obtained by minimizing relative gradients on the SGF; second, temporal consistency is proposed based on the adjacent frame similarity to fuse the adjacent dynamic saliency maps and get the fine motion saliency map; third, based on spatiotemporal consistency, salient objects are extracted by the fusion of the static saliency map and the motion saliency map adaptively. Experiments on the ViSal and SegtrackV2 public video saliency data sets show that, compared with the state-of-the-art image saliency methods and video sequence saliency object detection methods, the proposed algorithm can extract the salient object in the video sequence quickly, clearly, and accurately. It can be seen that the average F-score is close to 0.8, and the average mean absolute error (MAE) is about 0.06 on the ViSal data set, and on SegtrackV2, the average F-score is close to 0.7, and the MAE value is below 0.05, which indicates that the result of this algorithm is closer to the ground truth.