Scene-Based Priors for Bayesian Semantic Image Segmentation

Abstract

Based on the observation that semantic segmentation errors are partially predictable, this study proposes a compact formula using the confusion statistics of a trained classifier to refine (re-estimate) the initial label hypotheses. The proposed strategy is contingent upon computing the classifier confusion probabilities for a given dataset and estimating a relevant prior on the object classes present in the image to be classified. This study provides a procedure to robustly estimate the confusion probabilities and explore multiple prior definitions. Experiments are shown comparing performances on multiple challenging datasets using different priors to improve a state-of-the-art semantic segmentation classifier. The study demonstrates the potential to significantly improve semantic labeling and motivates future work for reliable label prior estimation from images.