Cloud Provisioning and Management with Deep Reinforcement Learning

Abstract

The first web applications appeared in the early nineteen nineties. These applica- tions were entirely hosted in house by companies that developed them. In the mid 2000s the concept of a digital cloud was introduced by the then CEO of google Eric Schmidt. Now in the current day most companies will at least partially host their applications on proprietary servers hosted at data-centers or commercial clouds like Amazon Web Services (AWS) or Heroku.\nThis arrangement seems like a straight forward win-win for both parties, the customer gets rid of the hassle of maintaining a live server for their applications and the cloud gets the customer’s business. However running a cloud or data-center can get expensive. A large amount of electricity is used to power the blades that inhabit the racks of the data-center as well as the air-conditioning that prevents those blades and their human operators from overheating. Further complications are added if a customer hosts in multiple locations. Where should incoming jobs be allocated too? What is the minimum number of machines that can run at each location that ensures profitability and customer satisfaction.\nThe goal of this paper is to answer those questions using deep reinforcement learning. A collection of DRL agents will take data from an artificial cloud environment and decide where to send tasks as well as how to provision resources. These agents will fall under two categories: task scheduling and resource provisioning.We will compare the results of each of these agents and record how they work with each other. This paper will show the feasibility of implementing DRL solutions for task scheduling and resource provisioning problems.