Cloud Data Management Systems: Opportunities and Challenges

Abstract

Cloud computing, an infrastructure of providing computing utility as a service, is changing a large part of the IT industry. By delivering to users "infinite" computing resources in a pay-as-you-go manner, cloud computing is turning into reality a dream that has long been cherished by the IT industry. For years, people have been hoping to achieve the abilities of continuously scaling up and of elastic resource utilizing, which now are within grasp. However, building a scalable data management system on existing commercial cloud platforms, such as Amazon EC2, poses a grand challenge. Indeed, new application requirements and the underlying hardware environment affect every aspect of the data management system, from individual components to system architecture. In this talk, we will present the opportunities and challenges of developing a scalable cloud data management system. First, we examine the anatomy of a cloud data intensive system. Then, we present three main challenges posed by a scalable cloud data processing system, i.e., multitenancy architecture, high throughout low latency transactions, and high performance reliable query processing. We will discuss why existing large-scale distributed systems such as peer-to-peer systems, distributed data management systems and massive parallel processing systems may not be able to deliver sufficient scalability, fault tolerance and satisfactory performance at cloud scale. Finally, we discuss possible solutions, consider current practices, and speculate the future of cloud computing.