Fine-Grained Expression Recognition Model Based on Multi-Scale Hierarchical Bilinear Pooling Network

Abstract

Facial expressions are characterized by subtle differences between expression classes and significant changes within a class, which increases the difficulty of expression recognition.To address the problem, a neural network model is proposed based on multi-scale bilinear pooling.The global features of facial expressions are extracted by using three networks with different scales.Then a hierarchical bilinear pooling layer is introduced, and multi-scale cross-layer bilinear features of the same network and different networks are integrated to capture some feature relationships between different levels, thus enhancing the ability of the model to represent and recognize subtle features of facial expressions. Multilayer feature information is fused by layer deconvolution, so the loss of key features that occurs when the neural network extracts features through multiple convolution layers and the pooling layer is solved.The experimental results show that the proposed model achieves a 73.725% recognition accuracy on FER2013 and 98.82% on CK+public data sets, outperforming SPLM, CL, JNS and other facial expression recognition algorithms.