FGBRSN: Flow-Guided Gated Bi-Directional Recurrent Separated Network for Video Super-Resolution

Abstract

Video Super-Resolution (VSR) is the task of reconstructing high-resolution (HR) video sequences from low-resolution (LR) video sequences. Apart from spatial information of reference frames, temporal information of neighboring frames is also important for reconstruction. Current VSR methods usually take advantage of the temporal information through optical flow estimation and compensation. However, optical flow estimation is often inaccurate and difficult, which may result in artifacts or blurring in the reconstructed frames. In this paper, we propose a Novel Flow-guided Deformable Alignment Module (NFDAM) for frame alignment. In this module, a lightweight optical network FNet is designed to estimate optical flow as a coarse offset, which then guides the deformable convolution at the feature and image level. On this basis, we propose a Flow-guided Gated Bidirectional Recurrent Separated Network (FGBRSN) for VSR, in which a gated recurrent structure is designed to leverage long-term information and a multilevel residual fusion approach is used in upsampling module. Our experiments on public datasets have shown that the proposed method improves both quantitative evaluation and visual effects compared with the existing methods.