Image Multi-Inpainting via Progressive Generative Adversarial Networks

Abstract

Image inpainting aims to inpaint missing pixels of an image naturally and realistically. Previous deep learning approaches typically require specific design for different types of masks and cannot generalize well to multiple inpainting scenarios simultaneously. Thus on top of most common stroke-type mask approaches, we in this paper pro-pose a unified framework to handle multiple types of masks simultaneously (e.g. strokes, object shapes, extrapolation, dense and periodic grids et al). We address this problem by proposing a progressive learning scheme to an Semantic Aware Generative Adversarial Network (SA-PatchGAN). Specifically, the overall training proceeds in multiple stages with different type of mask inputs, so that the model can gradually generate an output image from coarse to fine with mask independent property. In our experiments, we show that this strategy yields a large performance gain compared to the single-scale learning methods. We also introduce additional semantic conditioning to the discriminator which encourage high quality local style statistics, and show that this approach is effective on a wider scenario/tasks and could better adapt to various types of mask. Our method produces promising results on various mask types using one single model.