PFDN: Pyramid Feature Decoupling Network for Single Image Deraining

Abstract

Restoring images degraded by rain has attracted more academic attention since rain streaks could reduce the visibility of outdoor scenes. However, most existing deraining methods attempt to remove rain while recovering details in a unified framework, which is an ideal and contradictory target in the image deraining task. Moreover, the relative independence of rain streak features and background features is usually ignored in the feature domain. To tackle these challenges above, we propose an effective Pyramid Feature Decoupling Network (i.e., PFDN) for single image deraining, which could accomplish image deraining and details recovery with the corresponding features. Specifically, the input rainy image features are extracted via a recurrent pyramid module, where the features for the rainy image are divided into two parts, i.e., rain-relevant and rain-irrelevant features. Afterwards, we introduce a novel rain streak removal network for rain-relevant features and remove the rain streak from the rainy image by estimating the rain streak information. Benefiting from lateral outputs, we propose an attention module to enhance the rain-irrelevant features, which could generate spatially accurate and contextually reliable details for image recovery. For better disentanglement, we also enforce multiple causality losses at the pyramid features to encourage the decoupling of rain-relevant and rain-irrelevant features from the high to shallow layers. Extensive experiments demonstrate that our module can well model the rain-relevant information over the domain of the feature. Our framework empowered by PFDN modules significantly outperforms the state-of-the-art methods on single image deraining with multiple widely-used benchmarks, and also shows superiority in the fully-supervised domain.