Leveraging regression models for rule based complex event processing

Abstract

In recent times, IoT devices have surged enormously, which creates a lot of raw data that is dynamic in nature, and processing it in real time to find useful information is challenging. Complex event processing involves the analysis of large volumes of real-time data to identify patterns and events of interest. These events are formed based on a predefined set of rules, and since rules are created by domain experts and for dynamic data, there is a requirement for a robust model that can eliminate manual intervention for rule generation. In this paper, to help the domain experts, a regression-based model is proposed so that more accurate decision-making can be performed by finding more robust event patterns. For regression-based rule implementation, three models are compared: logistic regression, ridge regression, and support vector machine. The models are trained using an IoT temperature dataset and tested using a synthetically generated dataset with the same set of parameters.The ridge regression performed best among all the models, with an accuracy, precision, recall, and f1 score of 99% 97% 96% 95% among all. The entire experiment was carried out with the apache flink ecosystem and pattern API.