Cluster aware storage resource provisioning in a data center

Abstract

Clustering solutions are frequently used in large enterprise and mission critical applications with high performance and availability requirements. This is achieved by deploying multiple servers for the purpose of load balancing or by having redundant servers for the purpose of high availability. Most cluster management tools focus on the availability of servers and IP connectivity, and rely on shared network storage like Network Attached Storage (NAS), Storage Area Network (SAN), etc. for storage resiliency. Modern enterprise storage systems are complex, with millions of possible configurations. For end-to-end performance and availability, the configuration of storage resources and interconnection fabric need to be carefully planned. Adhoc configuration may lead to poor availability, deteriorated performance or higher cost. In this paper, we discuss the challenges associated with provisioning storage resources in cluster environment and present a list of best practices and novel optimization strategies to make storage more resilient for clusters. We have build a tool called Casper that automates the process of planning, optimization and deployment of shared network storage for clusters. Casper distributes storage load across different storage nodes and across different network paths to ensure load balancing and prevent single point of failure.