A Scalable Attention Mechanism Based Neural Network for Text Classification

Abstract

Abstract In general, deep learning based text classification methods are considered to be effective but tend to be relatively slow especially for model training. In this work, we present a powerful, so-called “scalable attention mechanism”, which performs better than conventional attention mechanism in terms of both effectiveness and the speed of model training. Based on the scalable attention mechanism, we propose a neural network for text classification. The experimental results on eight representative datasets show that our method can obtain similar accuracy to state-of-the-art methods with training in less than 4 minutes on an NVIDIA GTX 1080Ti GPU. To the best of our knowledge, our method is at least twice faster than all the published deep learning classifiers.