Flow-Guided Deformable Attention Network for Fast Online Video Super-Resolution

Abstract

Real-time online video super-resolution (VSR) on resource limited applications is a very challenging problem due to the constraints on complexity, latency and memory foot-print, etc. Recently, a series of fast online VSR methods have been proposed to tackle this issue. In particular, attention based methods have achieved much progress by adaptively aligning or aggregating the information in preceding frames. However, these methods are still limited in network design to effectively and efficiently propagate the useful features in temporal domain. In this work, we propose a new fast online VSR algorithm with a flow-guided deformable attention propagation module, which leverages corresponding priors provided by a fast optical flow network in deformable attention computation and consequently helps propagating recurrent state information effectively and efficiently. The proposed algorithm achieves state-of-the-art results on widely-used benchmarking VSR datasets in terms of effectiveness and efficiency. Code can be found at https://github.com/IanYeung/FastOnlineVSR.