Budget-Minimized Resource Allocation and Task Scheduling in Distributed Grid/Clouds

Abstract

The need for large-scale computing, storage and network capabilities by the scientific or business community has resulted in the development of cloud networks. Grid/Clouds users are provided with IT infrastructure (servers, storage, networks, etc.) as services called Infrastructure as a Service (IaaS). In this case, an efficient resource scheduling mechanism for allocating the infrastructure resources across the network will improve the resource efficiency in the cloud significantly. In this paper, we investigate the budget optimization of joint resources (storage, processor and network) allocation for IaaS model in distributed Grid/Clouds from the consumer's perspective. We develop a Mixed Integer Linear Programming (MILP) formulation along with a new resource model and propose a Best-Fit heuristic algorithm with different job scheduling policies. Our goal is to minimize the expenditure for each user to obtain enough resources to execute their submitted jobs, while enabling the Grid/Cloud provider to accept as many job requests from the users as possible. Both MILP and heuristic are tested on a 10- node topology and the Google Datacenter topology. The results show that the heuristic method can achieve approximate optimal solutions to MILP; it can reduce the user expense by at least 30%. In addition, Best-Fit algorithm with SSF (simple job structure first) job scheduling policy has the lowest blocking rate, which is 5%~25% less than other job scheduling policies.