A multiscale compressed video saliency detection model based on ant colony optimization

Abstract

In this paper, a novel multiscale visual saliency detection algorithm combining spatiotemporal features and ant colony optimization is proposed. In the method, both the spatial information, such as luminance, chrominance and texture, and the temporal information, namely motion, are used to fulfill a better prediction of visual saliency. Besides, the information we use in the method are all extracted directly from the compressed video bitstreams to avoid the time-consuming decompressing process. The concept of multiscale is introduced. We use graphs of different scales constructed by dividing the video frames into blocks of different sizes to achieve more human-eye adaptability. Then the spatial features, namely luminance, chrominance and texture, are extracted directly from discrete cosine transform coefficients while the temporal information are extracted from the motion vectors to form the heuristic matrices. Next, the heuristic matrixes are used as part of the ant colony optimization process. Each heuristic matrix is used to steer the ants in the algorithm and the ants deposit pheromone on the graph. The pheromone is updated through attenuation and evaporation thus forming spatial/temporal saliency maps. Finally, the spatial and temporal saliency maps of each scale are fused together through adaptive fusion, and maps of different scales are fused through linear fusion. Since the model is constructed using information in compressed domain individually, the decompression process is avoided to save more time and to be suitable for videos transmitted on the network. Besides, the proposed method has been extensively tested on several video databases with sequences in various scenes. Through experiments it can be seen that in both quantitative evaluation scores and intuitive visual effects, the algorithm in this paper exhibits a better performance compared to the contrast methods in this paper.