Efficient query processing on distributed stream processing engine

Abstract

Distributed stream processing engines, such as Storm and Samza, have been developed to process large scale stream data. The engines are scale out horizontally with shared nothing architecture, but they do not provide high-level query language like SQL. Supporting query language for flexible analysis has become an important issue. In this paper, we provide efficient continuous relational query processing on distributed stream processing engine. We propose a methodology to transform queries executable in the engine and optimization technique for query processing. Our experimental results show that our methodology is efficient on processing queries for data streams.