Routing in optical transport networks with deep reinforcement learning

Abstract

Deep reinforcement learning (DRL) has recently revolutionized the resolution of decision-making and automated control problems. In the context of networking, there is a growing trend in the research community to apply DRL algorithms to optimization problems such as routing. However, existing proposals fail to achieve good results, often under-performing traditional routing techniques. We argue that the reason behind this poor performance is that they use straightforward representations of networks. In this paper, we propose a DRL-based solution for routing in optical transport networks (OTNs). Contrary to previous works, we propose a more elaborate representation of the network state that reduces the level of knowledge abstraction required for DRL agents and easily captures the singularities of network topologies. Our evaluation results show that using our novel representation, DRL agents achieve better performance and learn how to route traffic in OTNs significantly faster compared to state-of-the-art representations. Additionally, we reverse engineered the routing strategy learned by our DRL agent, and as a result, we found a routing algorithm that outperforms well-known traditional routing heuristics.