Facial Expression Synthesis and Recognition with Pre-Trained StyleGAN

Abstract

Deep learning based facial expression recognition requires large-scale training data. However, the existing available expression datasets don't have enough labelled data. To overcome the above issue, we propose an expression synthesis model which uses the StyleGAN pre-trained on a large-scale face dataset to synthesize expression images with new identities, so as to increase the image diversity of expression datasets and improve the performance of expression recognition. First, expression images are projected into the latent space of StyleGAN by GAN inversion to obtain their latent codes. Second, the identity information learned by StyleGAN from the large-scale face dataset is transferred into expression images by manipulating the latent codes. An intra-class loss is introduced to reduce the data bias between synthesized images and real expression images, and the label smoothing regularization is introduced into cross entropy loss to consider the distribution of non-ground truth classes in synthesized images, so as to improve the performance of expression recognition. Experiments on both in-the-lab and in-the-wild datasets demonstrate that our approach is competitive to several state-of-the-art methods.