Intelligent Routing based on Deep Reinforcement Learning in Software-Defined Data-Center Networks

Abstract

In software-defined data-center networks, Elephant flow/Mice flow/Coflow coexist and multiple resources (bandwidth, cache and computing) coexist. However, the conventional routing methods cannot overcome the large gap between different performance requirements of flow and efficient resource allocation. Therefore, this paper proposes DRL-R (Deep Reinforcement Learning-based Routing) to bridge this gap. First, we recombine multiple resources (node's cache, link's bandwidth) by quantifying the contribution score of them reducing the delay. This actually converts the performance requirements of flow into resource requirements of this flow, hence, the routing problem can be converted into a job-scheduling problem in resource management. Second, a DRL agent deployed on an SDN controller continually interacts with the network for adaptively performing reasonable routing according to the network state, and optimally allocating network resources for traffic. We employ Deep Q-Network (DQN) and Deep Deterministic Policy Gradient (DDPG) to build the DRL-R. Finally, we demonstrate the effectiveness of DRL-R through extensive simulations. Benefitted from continually learning with a global view, DRL-R can improve throughput highest up to 40% and flow completion time highest up to 47% over OSPF. DRL-R can improve the load balance of link highest up to 18.8% over OSPF. Additionally, DDPG has better performance than DQN.