Contextual Information Aggregation and Multi‐Scale Feature Fusion for Single Image De‐Raining in Generative Adversarial Networks

Abstract

ABSTRACT Aiming to address issues such as non‐uniform rain density and misjudgment caused by noise in image de‐raining, we propose a single‐image de‐raining method based on a generative adversarial network with contextual information aggregation and multi‐scale feature fusion. First, we design a generator composed of encoding, context information aggregation, and decoding stages. Features are extracted using convolution, while expansion convolution effectively aggregates context information. Transposition convolution is then used to restore the image, enhancing the model's ability to perceive image details and achieve accurate image information judgment and content reconstruction. Second, we design a multi‐scale feature fusion discriminator structure to capture different image details using convolution kernels of different scales and connect feature maps from different scales. This improves the model's ability to understand image details and differentiate between authentic and fake images. Finally, we propose a new refinement loss function to reduce grid artifact generation and add Lipschitz constraints to further minimize the imaging gap. In this paper, peak signal‐to‐noise ratio and structural similarity are used as evaluation criteria, and experiments conducted on real and synthesized rain maps demonstrate the superior rain removal performance of the proposed method.