Document-Level Sentiment Analysis Using Attention-Based Bi-Directional Long Short-Term Memory Network and Two-Dimensional Convolutional Neural Network

Abstract

Due to outstanding feature extraction ability, neural networks have recently achieved great success in sentiment analysis. However, one of the remaining challenges of sentiment analysis is to model long texts to consider the intrinsic relations between two sentences in the semantic meaning of a document. Moreover, most existing methods are not powerful enough to differentiate the importance of different document features. To address these problems, this paper proposes a new neural network model: AttBiLSTM-2DCNN, which entails two perspectives. First, a two-layer, bidirectional long short-term memory (BiLSTM) network is utilized to obtain the sentiment semantics of a document. The first BiLSTM layer learns the sentiment semantic representation from both directions of a sentence, and the second BiLSTM layer is used to encode the intrinsic relations of sentences into the document matrix representation with a feature dimension and a time-step dimension. Second, a two-dimensional convolutional neural network (2DCNN) is employed to obtain more sentiment dependencies between two sentences. Third, we utilize a two-layer attention mechanism to distinguish the importance of words and sentences in the document. Last, to validate the model, we perform an experiment on two public review datasets that are derived from Yelp2015 and IMDB. Accuracy, F1-Measure, and MSE are used as evaluation metrics. The experimental results show that our model can not only capture sentimental relations but also outperform certain state-of-the-art models.