Facial Expression Transfer Based on Conditional Generative Adversarial Networks

Abstract

With the development of computer vision and image transfer, facial expression transfer has been more and more widespread applications. But there are still some problems, such as lack of realistic expression, poor retention of facial identity features and low synthesis efficiency. In order to solve the problems of facial expression transfer, the paper proposes a facial expression transfer model based on conditional generative adversarial network, which can generate a highly realistic face image with source facial expression and target facial identity features, when gave a source face image and a target face image. The model consists of two parts: the facial feature point fusion module and the expression transfer module. Among them, the facial feature point fusion module uses an auto-encoder to encode the face key feature point image of the source facial expression and the face feature key point image of the target face, so as to transfer the source facial expression information to the corresponding face key feature points of the target image; the expression transfer module uses the facial feature point fusion module to generate the face key feature point image and the target face image, and then generates an image with the source facial expression and the target face identity features through the modified U-net network. The model is finally validated on two publicly available datasets, RaFD and CK+, and the experimental results show that the generated facial expression is more realistic than the pix2pix model, and the model only needs to be trained once to complete the transfer between any facial expression.