Multimodal Emotion Recognition Using Deep Learning Techniques

Abstract

Humans have the ability to perceive and depict a wide range of emotions. There are various models that can recognize seven primary emotions from facial expressions (joyful, gloomy, annoyed, dreadful, wonder, antipathy, and impartial). This can be accomplished by observing various activities such as facial muscle movements, speech, hand gestures, and so forth. Automatic emotion recognition is a significant issue that has been a hotly debated research topic in recent years. At the moment, several research people have taken a component in inheriting or extra multimodal for higher understanding. This paper indicates a method for emotion recognition that makes use of 3 modalities: facial images, audio indicators, and text detection from FER and CK+, RAVDESS, and Twitter tweets datasets, respectively. The CNN model achieved 66.67 percent on the FER-2013 dataset of labeled headshots while on the CK+ dataset, 98.4 percent accuracy was obtained. Finally, diverse fusion strategies had been approached, and each of those fusion techniques gave distinctive results. This project is a step towards the sense of interaction between human emotional aspects and the growing technology that is the future of development in today's world.