Triplet Spatiotemporal Aggregation Network for Video Saliency Detection

Abstract

The effective aggregation of spatiotemporal information to accommodate real-world complex scenes is a fundamental issue in video saliency detection. In this paper, we propose a Triplet Spatiotemporal Aggregation Network (TSAN) to address it from the aggregation of spatiotemporal interaction, spatiotemporal information distribution, and multi-level spatiotemporal features. Firstly, we propose an interactive aggregation gate (IAG) module to model spatial and temporal global context information and perform inter-modal information transfer. Secondly, we employ an information distribution consistency (IDC) module to enhance the consistency of spatiotemporal representation by maximizing the correlation of spatiotemporal high-level features. Finally, we design a multi-level spatiotemporal feature aggregation (MSF) framework to merge cross-level and cross-modal features. These three modules are combined into a unified framework to jointly optimize spatiotemporal information for more precise results. Experimental results on five prevailing datasets show that TSAN outperforms previous competitors.