Semi-supervised machine learning for textual anomaly detection

Abstract

Anomaly detection comprises the identification of observations which do not follow the expected patterns of the assumed data set. We attempt to simplify the problem of textual anomaly detection by constructing a Multinomial Naïve Bayes classifier and enhancing it with an augmented Expectation Maximization (EM) algorithm. By doing so, we utilize large amounts of unlabelled data and show how the EM algorithm could increase the accuracy of the Naïve Bayes classifier. The process is applied to a binary classification environment in order to detect anomalies in text.