Emotion recognition from facial expressions using hybrid feature descriptors

Abstract

Here, a hybrid feature descriptor‐based method is proposed to recognise human emotions from their facial expressions. A combination of spatial bag of features (SBoFs) with spatial scale‐invariant feature transform (SBoF‐SSIFT), and SBoFs with spatial speeded up robust transform are utilised to improve the ability to recognise facial expressions. For classification of emotions, K ‐nearest neighbour and support vector machines (SVMs) with linear, polynomial, and radial basis function kernels are applied. SBoFs descriptor generates a fixed length feature vector for all sample images irrespective of their size. Spatial SIFT and SURF features are independent of scaling, rotation, translation, projective transforms, and partly to illumination changes. A modified form of bag of features (BoFs) is employed by involving feature's spatial information for facial emotion recognition. The proposed method differs from conventional methods that are used for simple object categorisation without using spatial information. Experiments have been performed on extended Cohn–Kanade (CK+) and Japanese female facial expression (JAFFE) data sets. SBoF‐SSIFT with SVM resulted in a recognition accuracy of 98.5% on CK+ and 98.3% on JAFFE data set. Images are resized through selective pre‐processing, thereby retaining only the information of interest and reducing computation time.