Text Classification under Attention Mechanism Based on Label Embedding

Abstract

Text classification is one of key tasks for representing the semantic information of documents. There have been a lot of studies that applied Long Short-Term Memory (LSTM) to this task. This paper develops a novel model to improve the traditional LSTM for text classification, which is based on label embedding and attention mechanism. We use Convolutional Neural Network (CNN) to learn the compatibility between words and labels, which is used as attention value to improve the sequence output generated by Bi-LSTM to ensure that the weight of related words is higher than that of unrelated words. Experimental results on five text datasets demonstrate that the proposed architecture is very competitive compared with the state-of-the-art models.