Improved conditional diffusion model for image super‐resolution

Abstract

Abstract Diffusion models have achieved remarkable success in image super‐resolution by addressing issues such as transition smoothing, insufficient high‐frequency information, and training instability encountered in regression‐based and GAN‐based models. However, challenges persist when applying diffusion models to image super‐resolution, including randomness, inadequate conditional information, high computational costs, and network architecture complexities. In this article, the authors introduce a diffusion model based on Mean‐Reverting Stochastic Differential Equations (SDE), and propose the use of ENAFBlocks instead of traditional ResBlocks to enhance model performance in noise prediction. The Mean‐Reverting SDE effectively mitigates the randomness of the diffusion model by leveraging low‐resolution images as means. Additionally, an LR Encoder is introduced to capture hidden information from LR images, providing a more robust condition for stable result generation by the noise predictor. To efficiently handle high‐resolution images within limited GPU memory, the method employs adaptive aggregate sampling, which merges overlapping regions smoothly using weighted averaging. Furthermore, color variations are addressed during diffusion model sampling through color correction. Extensive experiments on CelebA, DIV2K, and Urban100 demonstrate that the method outperforms state‐of‐the‐art diffusion models like IDM, with a PSNR improvement of 0.22 dB, FID reduction of 2.35, and LPIPS reduction of 0.05 on the DIV2K dataset, along with a reduced parameter count and faster inference time.