Single image super-resolution using capsule generative adversarial network

Abstract

The current research aims to investigate and propose a Generative Adversarial Network (GAN) architecture [53] using capsule network architecture [76] in the discriminator module of the proposed model (Caps-GAN) for Single Image Super-Resolution. Besides, the study aims to develop the proposed SR framework in three scale factors. Finally, the performance of Caps-GAN is compared with other state-of-the-art models. Our Caps-GAN model consists of three fundamental components: the generator module, capsule discriminator module, and combinations of loss functions based on the GAN concept. The proposed generator utilizes the residual in residual dense blocks (RRDB) architecture [28] under a progressively up-sampling framework [30]. At the same time, the depth-wise bottleneck projections concept [38] is employed to transfer the high-frequency details of the early layer to each up-sampling stage to prevent gradient vanishing. Additionally, a novel fusion objective function that combines Multi-level SSIM loss and L2 loss (MS-SSIM + L2) is introduced to improve the quantitative and qualitative results and reconstruct the sophisticated details. In our Caps-GAN model, the CNN-based discriminator has been replaced with the capsule network architecture. Duo to the capability of the capsule network to extract the hierarchical feature relationships, our capsule discriminator demonstrates superior performance in extracting difficult-to-learn patterns in training our model. This capability leads to training our GAN model much better and faster than the CNN-based discriminator. The capsule discriminator is trained with GAN loss [28], and the generator is trained with a perceptual loss [8]. Our perceptual loss consists of two types of losses including a content loss (pre-trained model) for producing the overall appearance of the image, and an adversarial loss for producing high-frequency details of texture. The quantitative and visual evaluations are based on five benchmark datasets including, Set5, Set14, BSDS100, Urban100, Manag109, and DIV2K. For quantitative comparison, the quality metrics including PSNR and SSIM, and the MOS test for visual comparison at two scales.