BESRGAN: Boundary equilibrium face super‐resolution generative adversarial networks

Abstract

Abstract Existing Generative Adversarial Networks (GAN)‐based face hallucination algorithms are hard to control the face fidelity of the generated samples, and easily generate flawed faces with unfavourable artefacts and distortions. To address this problem, the authors propose a fidelity‐controllable face super‐resolution (FSR) network boundary equilibrium face super‐resolution generative adversarial networks (BESRGAN), a fidelity ratio is introduced in their network to control how much the adversarial effect the discriminator is put on the generator; therefore, the authors’ network better trades off the objective and perceptual quality. Additionally, the authors design an equilibrium perceptual discriminator to match the perception loss distributions. Under the equilibrium constraint, the discriminator pays more attention to learning fine‐grained feature statistics of ground truths, and further guides the generator to produce photo‐realistic faces, especially in terms of facial textures. Moreover, the authors propose a novel channel‐spatial attention module (CSAM) to eliminate local distortions, by further fusing richer information from the facial prior knowledge and global high‐level facial descriptions. Extensive experiments illustrate that the authors’ approach preserves high pixel‐wise accuracy while achieving superior visual performance against state‐of‐the‐art methods. Specifically, the peak signal to noise ratio (PSNR) and structural similarity index (SSIM) of the authors’ proposed BESRGAN rise 0.64 dB and 0.02 for CelebA compared with one of the state‐of‐the‐art face super‐resolution (FSR) methods.