Multi-branch Attention Consistency Network for Facial Expression Recognition

Abstract

Due to the high inter-class similarity and subjective annotation of facial expressions, annotation uncertainty has become the key challenge in recent years. In this paper, we propose a Multi-branch Attention Consistency Network for facial expression recognition by combining latent label distribution learning and attention consistency to alleviate the annotation uncertainty. To be specific, we design three modules, namely multi-branch feature classification (MFC), multi-branch latent distribution learning (MLD) and multi-class attention consistency (MAC). The MFC classifies uncertain expressions through multiple auxiliary branches, which obtains attention maps and the degree of confidence for different facial categories. The MLD guides the target branch to learn latent label distributions from auxiliary branches. The MAC learns attention regions by multi-class attention consistency between auxiliary and target branches. Finally, we demonstrate the effectiveness of our proposed method by conducting experiments on three popular facial expression datasets. Experimental results show that our method achieves the state-of-the-art results of 90.16%, 89.98%, 63.12% accuracy on RAF-DB, FERPlus and AffectNet datasets, respectively.