Multi-feature audio-visual person recognition

Abstract

We propose a high-performance low-complexity audio-visual person recognition framework suitable for on-line user authentication for various web-applications which delivers robustness against various types of imposter attacks by capturing face and speech dynamics from the video of the user. Instead of using the traditional frontal-face image, a set of compressed face profile vectors are extracted from multiple poses of the person. Similarly, multiple user-selected passwords are used to create robustness against imposter attacks. A novel FGRAM-CFD speech feature is proposed which captures the identity of the user from the speech dynamics contained in the password. The novel signal processing methods proposed here for speech and face feature-extraction led to high discriminative power of the combined audio-visual features. This allowed the classifier to remain simple, yet delivering a reasonably high performance at significantly low complexity as demonstrated by our trials on a 210-user audio-visual biometric database created for this research.