Structured prediction for urban scene semantic segmentation with geographic context

Abstract

In this work we address the problem of semantic segmentation of urban remote sensing images into land cover maps. We propose to tackle this task by learning the geographic context of classes and use it to favor or discourage certain spatial configuration of label assignments. For this reason, we learn from training data two spatial priors enforcing different key aspects of the geographical space: local co-occurrence and relative location of land cover classes. We propose to embed these geographic context potentials into a pairwise conditional random field (CRF) which models them jointly with unary potentials from a random forest (RF) classifier. We train the RF on a large set of descriptors which allow to properly account for the class appearance variations induced by the high spatial resolution. We evaluate our approach by an exhaustive experimental comparisons on a set of 20 QuickBird pansharpened multi-spectral images.