Unsupervised Domain Adaptation for Building Extraction of High-Resolution Remote Sensing Imagery Based on Decoupling Style and Semantic Features

Abstract

When the buildings themselves or the environments they are located in changes, it poses a cross-domain problem for building extraction. Since the semantics of same category should be consistent across domains, a memory mechanism can be used to drive models to capture it. A key prerequisite for a satisfactory memory mechanism is that the memory content is highly task-relevant, i.e., the memory mechanism represents domain-invariant semantic features of buildings as much as possible. It is worth noting that images from different domains have diverse styles, which can interfere with the representation of domain-invariant semantic features of buildings. To maximize the effectiveness of memory mechanisms in cross-domain building extraction tasks, it is necessary to reduce the interference of style information. Therefore, we propose a cross-domain extraction method for buildings based on decoupling style-semantic feature. It expresses the image style features and building semantic features separately and optimizes the domain-invariant features of buildings by constraining their relationship through an orthogonal loss. Specifically, on the one hand, style features within the same domain as well as semantic features within and across domains are narrowed down through contrastive learning to ensure the consistency of style features extracted within the same domain and the consistency of semantics across domains. On the other hand, the style features and domain-invariant semantic features in memory mechanism can be stored and reused to fully exploit self-supervised information. Results of the cross-domain experiments show that the proposed method can achieve optimal building extraction.