An integrated CycleGAN-Diffusion approach for realistic image generation

Abstract

Cycle-Consistent Adversarial Network (CycleGAN) has been pivotal for image style transfer with the realm of computer vision. However, the quality of CycleGAN's output often lacks fine-grained details when generating realistic images. Therefore, this paper proposes a realistic image generation approach based on an integrated CycleGAN-Diffusion network to achieve higher image quality with a comparative small model. To evaluate the Diffusion model's ability to produce high-fidelity images under resource constraints and to compare its image quality with datasets processed using CycleGAN, we apply irregular masks with Gaussian noise during the inpainting and restoration phases. For assessing sample fidelity, we utilizes Mean Squared Error (MSE), Inception Score (IS) and Fréchet inception distance (FID) in this paper. Through extensive experiments, the proposed network is proved to perform better in generating high fidelity images, helping us to achieve FID scores of 5.98 and IS scores of 8.34 at 160×160 resolutions in the process of restoration and achieve IS scores of 7.69 in the inpainting process, both outperforming previous CycleGAN.