Text Classification Based on Convolutional Neural Network and Attention Model

Abstract

Using the traditional convolutional neural network (CNN) model for text classification, it is difficult to effectively capture the important local features in the text and the correlation between the feature words in the input text and the text. In order to solve this problem, this paper introduces the attention mechanism into the basic CNN model and establishes three CNN-based architectures: ATCNN-1, ATCNN-2 and ATCNN-3. ATCNN-1 increases the attention mechanism layer after the word embedding layer to obtain important local feature words, thus improving the features of convolution calculation. ATCNN-2 introduces the attention mechanism after the convolutional layer, and uses the attention mechanism to calculate the weights of each convolutional output vector to distinguish the importance degree, so that the model can extract features selectively. ATCNN-3 superimposes ATCNN-1 and ATCNN-2 together, combining the advantages of ATCNN1 and ATCNN-2. The experimental results in the two tasks of emotion classification and Chinese news text classification show that the effects of ATCNN-1, ATCNN-2 and ATCNN-3 are obviously better than the traditional CNN model, and the classification effect is also improved to some extent compared with the classical text classification model.