A Task Scheduling Approach for Real-Time Stream Processing

Abstract

To solve two main problems in task scheduling algorithms for real-time stream processing: the lack of consideration for links between tasks and the lack of support for dynamic scenes, this paper presents a new task scheduling approach. First, it concludes common scheduling scenarios on platforms for real-time stream processing. Second, it proposes matrix model of describing the real-time task scheduling problem. Third, it describes the specific scheduling approach and the algorithm set in it, and finally it implements the scheduling approach on SpeedStream, a real-time stream processing platform. Results prove that the approach can not only effectively balance the load of cluster nodes but also reduce communication traffic passing through switches. This approach is suitable for deployment and use in large-scale clusters.