Cross-spectral face recognition in heterogeneous environments: A case study on matching visible to short-wave infrared imagery

Abstract

In this paper we study the problem of cross spectral face recognition in heterogeneous environments. Specifically we investigate the advantages and limitations of matching short wave infrared (SWIR) face images to visible images under controlled or uncontrolled conditions. The contributions of this work are three-fold. First, three different databases are considered, which represent three different data collection conditions, i.e., images acquired in fully controlled (indoors), semi-controlled (indoors at standoff distances ≥ 50m), and uncontrolled (outdoor operational conditions) environments. Second, we demonstrate the possibility of SWIR cross-spectral matching under controlled and challenging scenarios. Third, we illustrate how photometric normalization and our proposed cross-photometric score level fusion rule can be utilized to improve cross-spectral matching performance across all scenarios. We utilized both commercial and academic (texture-based) face matchers and performed a set of experiments indicating that SWIR images can be matched to visible images with encouraging results. Our experiments also indicate that the level of improvement in recognition performance is scenario dependent.