Three language political leaning text classification using natural language processing methods

Abstract

In this article, the problem of political leaning classificationof the text resource is solved. First, a detailed analysis of ten stud-ies on the work’s topicwas performed in the form of comparative characteristicsof the used methodologies.Literary sources were compared according to the problem-solvingmethods,the learning that was carried out, the evaluation metrics, and according to the vectorizations.Thus, it was determined that machine learning algorithms and neural networks, as well as vectorizationmethods TF-IDF and Word2Vec, were most often used to solve the problem.Next, various classification models of whether textual information is pro-Ukrainian or pro-Russian were built based on a dataset containing messages from social media users about the events of the large-scale Russian invasion of Ukraine from February 24, 2022.The problem was solved with the help of Support Vector Machines, Decision Tree, Random Forest, Naïve Bayes classifier,eXtreme Gradient BoostingandLogistic Regressionmachine learning algo-rithms, Convolutional Neural Networks, Long short-term memory and BERT neural networks, techniques for working with unbal-anced dataRandom Oversampling, Random Undersampling , SMOTE and SMOTETomek, as well as stacking ensembles of models.Amongthe machine learning algorithms, LR performed best, showing a macro F1-scorevalue of 0.7966 when features were trans-formed by TF-IDF vectorization and 0.7933 when BoW.Among neural networks, the best macro F1-scorevalue of 0.76was ob-tained using CNN and LSTM.Applying data balancing techniques failed to improve the results of machine learning algorithms.Next, ensembles of models from machine learning algorithms were determined. Two of the constructed ensembles achieved the same macro F1-scorevalue of 0.7966 as with LR. Ensembles that wasable to do so consisted of the TF-IDF vectorization, the B-NBC meta-model, and the SVC, NuSVC LR, and SVC, LR base models, respectively.Thus, three classifiers, the LR machine learning algorithmand two ensembles of models, which were defined as a combination of existing methods of solving the problem, demon-strated the largest macro F1-score value of 0.7966. The obtained models can be used for a detailed review of various news publica-tions according to the political leaning characteristic, information about which can help people identify being isolated by a filter bubble.