Micro-expression Recognition based on Bimodal Contrastive Learning

Abstract

Micro-expressions are brief, subtle and involuntary facial gestures, which usually hide real emotions of people and are difficult to capture. Considering that micro-expressions data samples contain distinctive features of specific categories and common features of different categories, a micro-expressions recognition network based on bimodal contrastive learning is proposed in this paper. The network mainly includes bimodal feature extraction module, bimodal contrastive learning fusion module and classification and recognition module. First, the micro-expressions sequence is divided into RGB sequence and optical flow sequence, and the loss between them is constructed by contrastive learning. The network extracts bimodal common features. Second, in order to extract distinctive features, bimodal features are fused and label data is used to optimize the network. The network extracts bimodal features, while extracting distinctive features of different categories. The results show the superiority of the proposed method over other state-of-the-art methods for 5 categories of micro-expressions on CASME II, SAMM and 6 categories of micro-expressions on MMEW.