Real-Time Semantic Web Data Stream Processing Using Storm

Abstract

Semantic web technologies are increasingly used for the management of data flows. Several RDF flow processing systems have been proposed. The data at the entry of the system is big and generated continuously at a fast and variable rate. As a result, storing and processing the entire flow becomes costly and reasoning almost impossible. Consequently, the use of techniques allowing to reduce the load while preserving the semantics of the data, makes it possible to optimize the treatments even the reasoning. However, none of the SPARQL extensions include this functionality. Thus, In this paper we present a system for managing RDF data flows in real-time, the system contains two parts, the first manages the storage of RDF data and the second process the data that comes in real-time, and combines these news data with old ones to respond to requests from users, programs, and software agents. For validation, this approach makes it possible to detect events and automatically extract relations from them, in RDF format. To do this, the system analyzes Twitter messages in real-time simultaneously with the processing of RDF data stored in a triplestore.