Deep Learning based Dynamic Object Probability with Long Short-Term Memory for Drowsiness Detection

Abstract

Driver Drowsiness Detection (DDD) is crucial for preventing road accidents and serves as a pivotal element for shift from semi-autonomous to fully autonomous driving. Various Deep Learning (DL) models leveraging video sequences, offer sophisticated motion sensing capabilities. In this paper, the potential of identifying the regions of the eyes and mouth to capture yawning behavior, serving as a reliable indicator of drowsiness. The proposed Dynamic Object Probability with Long Short-Term Memory (DOP-LSTM) technique is employed to specifically detect the driver within the video sequence. Subsequently, the Dense Scale-Invariant Feature Transform (D-SIFT) technique is utilized to extract the regions of the driver's eyes and mouth. The features extracted from both the eyes and mouth are fused using Multi-Scale Feature Fusion (MSFF) techniques to determine the driver's state. LSTM is implemented for classification and detection and it demonstrates superior performance, achieving Precision of 99.69%, F-Measure of 99.02%, Recall of 98.30%, and Accuracy of 98.81%, when compared to existing models such as Inception V3-lstm, Hybrid-DDD, and Multi-scale Convolutional Neural Network-Flamingo Search Algorithm (MCNN-FSA).