UNet Based Multi-Scale Recurrent Network for Lightweight Video Deblurring

Abstract

With the recent widespread use of smartphones and digital video cameras, the opportunities to handle digital video have increased significantly. However, despite improvements in the performance of the hardware, the captured video often contains information that is not necessary for the purpose of the video. In particular, factors such as camera shake and object movement can cause blurring in video. So, we propose a method to deblur a video by software processing of the video after shooting. In conventional methods, video deblurring is often performed using a network whose main task is video super-resolution. However, in super-resolution, the size of the input and output images are different, while the input and output images are the same size in deblurring. In the case of deblurring, the input images are input to the network after a simple downsampling process, which is not optimized for the same size as the input and output images. Therefore, the proposed method constructs a multi-scale network based on UNet. The UNet-based network is a successful method for single image deblurring. Because a video is a sequence of multiple images, we use a method that has been successful in single image deblurring. Furthermore, we add improvements to the network based on the structure of MPRNet. The feature extraction modules of the bottom and the second stage of the network are replaced with a single-stage UNet. These improvements resulted in a 34.80dB in PSNR and 0.973 in SSIM on the GoPro dataset despite about 75% of the FLOPs of BasicVSR++ and 3% of the FLOPs of VRT. On the DVD dataset, the proposed model achieved 34.36dB in PSNR and 0.966 in SSIM. Further ablation studies show the effectiveness of various components in our proposed model.