Text classification problems via BERT embedding method and graph convolutional neural network

Abstract

This paper presents a hybrid technique of combining the BERT embedding method and the graph convolutional neural network. This combination is then employed to solve the text classification problem. Initially, we apply the BERT embedding method to the whole corpus in order to transform all the texts into numerical vectors. Then, the graph convolutional neural network will be applied to these numerical vectors to classify these texts into their appropriate classes. Especially, in our approach, we need only a few labeled texts for the model training. For the illustration, in this paper, we use the BBC news and the IMDB movie reviews datasets to perform our experiments, showing that the performance of the graph convolutional neural network model is better than the performances of the combination of the BERT embedding method with other classical machine learning models.