Speech emotion recognition method based on time-aware bidirectional multi-scale network

Abstract

Abstract In response to the difficulty of traditional speech emotion recognition models in capturing long-distance dependencies in speech signals and the impact of changes in speaker pronunciation speed and pause time, this paper proposes a new time emotion modeling method called Time Perceived Bidirectional Multi-scale Network (TIM-Net), which is used to learn Multi-scale contextual emotion expression in different time scales. TIM-Net starts by acquiring temporal emotional representations using time-aware blocks. Subsequently, information from different time points is combined to enhance contextual understanding of emotional expression. Finally, it consolidates various Timescale features to better accommodate emotional fluctuations. The experiment shows that the network can focus useful information on features, and the WAR and UAR of TIM-Net are significantly better than other models on RAVDESS, EMO-DB, and EMOVO datasets.