Gated Bi-Recurrent Separate Network for Video Super-Resolution

Abstract

Video super-resolution (VSR) aims to reconstruct high-resolution (HR) video sequences from low-resolution (LR) video sequences by image processing, which has been widely applied to different scenarios, including ultra HD video display, object detection, and medical image reconstruction. VSR exploits both intra-frame information and inter-frame information and makes full use of spatial and temporal information to reconstruct images, thus outperforming single-image super-resolution (SISR). In this paper, a gated bi-recurrent separate network (GBSN) is proposed, which gets spatio-temporal information to ensure a fine and realistic reconstruction of each frame by utilizing the bi-recurrent coupled propagation structure. The gated structure is used to control feature updating, which limits the propagation of error features. For the bidirectional network, the low-frequency part of the image is reconstructed in forward and the corresponding high-frequency part is reconstructed in backward. A fusion module is further developed to integrate these two parts. Experiments have been conducted to evaluate the performance of the proposed method, where Vid4, UDM10, and Vimeo-90K-T datasets are adopted. As compared to the state-of-the-art VSR methods in the literature, the proposed method enhances the peak signal to noise ratio (PSNR) to 28.51dB on Vid4 and 39.84dB on UDM10, respectively.