HDC-Based Convolutional Neural Network Image Inpainting

Abstract

Recently deep learning-based methods have been widely used in various aspects of image processing, with remarkable results for the challenging task of filling large areas of missing images. These methods can generate visually plausible structures and textures, but fuzzy filling can also occur when irregular voids are encountered. This is mainly due to the grid effect caused by unreasonable expansion factor setting in the use of neural network and continuous expansion convolution. As a result, the deep convolution operation cannot effectively obtain the details of the initial feature map, resulting in unreasonable repairing effect. Based on this phenomenon, we propose a convolutional neural network structure based on HDC (hybrid dilated convolution). The first stage consists of ordinary convolution and continuous dilated convolution, and the design of the dilated factor in the continuous dilated convolution is following the HDC method, which makes full use of the underlying feature map information while increasing the perceptual field. The second stage of the network consists of a convolutional branch and a perceptual branch, which are in parallel structure. The perceptual branch generates the filled region by extracting the feature of interest from the known region and generating the filled region based on the feature, after which it is aggregated with the feature map generated by the convolutional branch and fed into the decoder and deconvoluted to produce the restored image. Experimental results show that our method produces higher-quality inpainting results in terms of generating coherence, clarity, and visual confidence.