Reinforcement learning algorithms for adaptive load-balancing for web applications

Abstract

This research investigates the application of reinforcement learning (RL) to optimise load balancing in Nginx web applications. We developed a simulation environment on AWS to evaluate three enhanced RL algorithms: Epsilon-greedy, Upper Confidence Bound, and Proximal Policy Optimization (PPO) against classic methods (round-robin and Least Connections) under diverse load conditions, including normal loads, burst loads, server failures, and heterogeneous server instances. Our results demonstrate that RL, particularly PPO, significantly outperforms classic methods. Notably, PPO achieved up to a 30% increase in throughput, a 20% reduction in latency, and a 5% improvement in the successful message rate compared to the best-performing classic algorithm. These improvements were most pronounced under challenging conditions such as burst loads and server failures, highlighting the adaptability and resilience of RL-based load balancing.