Encoding-Aware Deep Video Super-Resolution Framework

Abstract

Video super-resolution(VSR) upscales a low-resolution video to the higher one. Most applications require compression of the super-resolved video due to limited internet bandwidth and storage capacity. However, most studies on VSR techniques have focused only on improving image quality, ignoring the impact of the compression process on visual quality. Consequently, even a VSR with good visual quality has a risk of significant loss of quality when serviced online or stored as a file. To address this problem, we propose an encoding-aware VSR framework. In the framework, we created a differentiable virtual codec to estimate the bit rate and used it for the loss function, which optimizes the super-resolved videos by considering the rate-distortion trade-off relationship and eventually leads to the prevention of visual quality degradation. According to the results, our real-time VSR model for x4 upscaling, trained with 1,191K parameters, yields a maximum gain of 13.2% over state-of-the-art VSR models based on the Bjøntegaard delta rate.