Video Super-Resolution with Pyramid Flow-Guided Deformable Alignment Network

Abstract

Video super-resolution (VSR) aims to recover high-resolution frames from their multiple low-resolution counterparts. How to fully exploit the spatio-temporal information among video sequences for VSR is significant but challenging. The challenges can be summarized as: 1) How to perform frame alignment when encountering displacement and occlusion. 2) How to effectively utilize spatio-temporal information for performance improvement. To address the above core problems, this paper proposes a pyramid flow-guided deformable alignment network for VSR (PFDVR) to achieve precision frame alignment and efficient spatiotemporal feature exploitation. Specifically, a pyramid flow-guided deformable alignment module (PFGDA) is proposed to perform feature alignment in a coarse-to-fine manner, and we employ bidirectional recurrent feature propagation to excavate temporal information. To capture the long-term spatio-temporal dependency, we propose an omniscient progressive fusion module (OPF) to achieve multi-level feature fusion both in spatial and temporal dimensions. The experimental results have demonstrated that our PFDVR can achieve promising SR performance.