Efficient Microservice Deployment in the Edge-Cloud Networks With Policy-Gradient Reinforcement Learning

Abstract

The rise of user-centric design demands ubiquitous access to infrastructure and applications, \nfacilitated by the Edge-Cloud network and microservices. However, efficiently managing resource allocation \nwhile orchestrating microservice placement in such dynamic environments presents a significant challenge. \nThese challenges stem from the limited resources of edge devices, the need for low latency responses, and the \npotential for performance degradation due to service failures or inefficient deployments. This paper addresses \nthe challenge of microservice placement in Edge-Cloud environments by proposing a novel Reinforcement \nLearning algorithm called Bi-Generic Advantage Actor-Critic for Microservice Placement Policy. This \nalgorithm’s ability to learn and adapt to the dynamic environment makes it well-suited for optimizing \nresource allocation and service placement decisions within the Edge-Cloud. We compare this algorithm \nagainst three baseline algorithms through simulations on a real-world dataset, evaluating performance \nmetrics such as execution time, network usage, average migration delay, and energy consumption. The results \ndemonstrate the superiority of the proposed method, with an 8% reduction in execution time, translating \nto faster response times for users. Additionally, it achieves a 4% decrease in network usage and a 2% \ndecrease in energy consumption compared to the best-performing baseline. This research contributes by \nreproducing the Edge-Cloud environment, applying the novel Bi-Generic Advantage Actor-Critic technique, \nand demonstrating significant improvements over the state-of-the-art baseline algorithms in microservice \nplacement and resource management within Edge-Cloud environments.