Correlation aware synchronization for near real time decision support systems

Abstract

Many large companies, especially those in financial and insurance service sectors, approach the market with a decentralized management structure, such as by line of business or geographical market segments. However, these companies require access to distributed and possibly heterogeneous data sources for corporate level decision making. In this paper, we focus on challenges of supporting a decision support system (DSS) based on a hybrid approach (i.e. a federation system with replication of frequently accessed remote data sources) for time-sensitive agile business intelligence applications. The response time requirement (and a realistic goal) for such a DSS is near real time (i.e. 2 ~ 3 minutes to 20 ~ 30 minutes). The users of a DSS care about not only the response time but also the time stamp of the business operation reports since out-dated reports introduce uncertainty and risks to decision-making. Thus, the information value of a report decreases as time passes. We present a framework of correlation aware synchronization of replicas used in DSS to optimize information values of business reports as a whole. The framework exploits correlation of usage and synchronization latency of replicas in a single query and a workload of queries for an optimal synchronization schedule. We have conducted extensive evaluations based on both TPC-H and synthetic workload. The proposed correlation aware synchronization effectively improves up to 50% of information value comparing with fixed synchronization plans on average.