Image inpainting algorithm based on convolutional neural network structure and improved Deep Image Prior

Abstract

In previous studies, researchers believed that the reason for the excellent performance of convolutional neural networks was that they could learn hidden information from special-purpose datasets, and the emphasis was on learning. Recently, the authors of Deep Image Prior proved that the generator structure itself (using convolutional neural network) could extract image prior information and be used for the image inpainting task. In this paper, based on Deep Image Prior, four improvements (mix input, network noise, weight decay, and burning mean output) are proposed for preventing overfitting and improving output stability. Peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) in two stepwise comparative experiments showed that our image inpainting algorithm surpassed the original algorithm and state-of-the-art algorithms after adding the proposed improvements in sequence. In large hole inpainting, the PSNR of our algorithm was 3.23 dB higher than in the original Deep Image Prior. Then, in a binary Bernoulli inpainting experiment, our algorithm achieved better performance in most classical image inpainting, proving that the algorithm could use the same set of parameters for each image in the task. In addition, this experiment also illustrated the performance of burning mean output in stabilizing the output and reducing the influence of meaningless noise in the early stage of the iteration on subsequent image inpainting. Keywords: convolutional neural network; Deep Image Prior; image prior information; image inpainting; overfitting; large hole inpainting; binary Bernoulli inpainting DOI: 10.33440/j.ijpaa.20200304.135 Citation: Wang J S, Han Y H. An image inpainting algorithm based on convolutional neural network structure and improved Deep Image Prior. Int J Precis Agric Aviat, 2020; 3(4): 65–73.