Reinforcement Learning based Heterogeneous Resource Provisioning for Cloud Web Applications

Abstract

Cloud computing enables enterprises to deploy Web applications on elastically rented Virtual Machines (VMs). In order to minimize VM rental costs while fulfilling Service Level Agreements (SLAs), it is crucial to design effective VM provisioning algorithms. Queueing and control theories based methods are widely used in existing works. However, complex nonlinear system characteristics restrict these methods from getting well performance and most of them are tailored for homogeneous resources. In this paper, a deep reinforcement learning method is proposed for provisioning heterogeneous multi-type VMs which takes advantage of the deep-learning-network's nonlinear modeling ability, incremental-VM-capacity-based action space definition and Integer Linear Programming based mapping from the incremental capacity to the multi-type VM renting plan. Experimental results demonstrate that our approach achieves the fastest convergence speed and highest total rewards.