Architectural Design Of Data Stream-Based Big Data Real-Time Analysis System

Abstract

Conducting the RT analysis of large-scale data stream severely challenges the IT hardware facilities and system Architecture.To cope with the needs of large-scale RT analysis, the article analyzes and compares the mainstream open-source technology and then designs the system architecture that consists of data collection, data distribution, real-time computing, persistent storage, resource allocation and scheduling.