Deception Detection using a Multimodal Stacked Bi-LSTM Model

Abstract

In the scientific community, researchers have recently become interested in the automatic identification of deceptive actions because of the range of fields where it might be advantageous, including criminology or security. Deception detection in conversational speech has drawn much attention in recent years. Given the significant risks associated with trial outcomes, using precise and efficient computational methods to assess the accuracy of court evidence may be very beneficial throughout the decision-making process. This study discusses about spotting deception in trial data from actual cases. Doing this has some challenges associated with creating a robust model that can accurately classify the deception and perform this action as fast as possible. Due to the limited number of videos and datasets available, some models overfit the data. A model should exist which can classify the data using various modalities, i.e., video, audio and text, and be able to work on multiple different datasets with excellent accuracy. This study has used videos from actual trials that were collected from open court proceedings and some videos from other datasets. To design a robust deception detection system that discriminates between witnesses and defendants, genuine and fraudulent testimony, this study investigates the utilization of text, audio and video modalities. By extracting and integrating information about the spoken words from audio, this study can achieve an accuracy of 80% approximately. The proposed model results with a classification accuracy of 96% approximately in an extended approach to perform video transcriptions. The Bag-of-lies dataset, a multimodal database captured in real-world settings has achieved an accuracy of 85%. The Miami University Deception Detection Dataset focuses on people telling truths and lies about their social relationships, achieved an accuracy of 98.1 % on the presented model. The proposed model employs LSTM (Long-Short Term Memory), Bidirectional LSTM (Long-Short Term Memory), CNN (Convolution Neural Network), and RestNet50. The results demonstrate that the proposed algorithm performs better at detecting deception than humans.