Few-shot Remote Sensing Imagery Recognition with Compositionality Inductive Bias in Hierarchical Representation Space

Abstract

Remote sensing scenes from aerial perspective can be constructed by distinct visual parts in a combinatorial number of different ways. Such combinatorial explosion poses great challenges to understanding remote sensing imagery (RSI) with few prior instances (i.e., few-shot RSI recognition). Despite empirical success of existing methods such as data augmentation and knowledge transfer, no large-scale dataset can cover all possible combinations of visual parts. In this case, the prior knowledge learned from these data-driven methods may exhibit dataset bias, resulting in inadequate generalization to the current recognition task. Different from the naive data-driven strategies mentioned above, we alternatively devote to delicate feature modeling by constraining the mapping behavior of deep neural networks. Specifically, we embed inductive bias of compositionality into hierarchical latent representation space, which operates on two aspects: 1) disentangled and reusable representation. We establish a clustering-oriented factorized representation with a mixture model to represent multipart distributions of tokens. Each cluster centroid represents a re-occurring part. New patches are allocated to the nearest cluster centroid, and then we obtain the posterior representation; 2) compositional and discriminative representation. We introduce a hierarchical context prediction mechanism for compositional representation learning, utilizing a predictive NCE loss function to encourage global remote sensing scenes to accurately predict similar local parts, and thus automatically inferring compositional representations of high-level but discriminative latent concepts. Extensive experiments, including comparative experiments with SOTA, sensitivity evaluations, and ablation studies, demonstrate comparable or even superior performance of our method in few-shot RSI recognition.