Efficient Elitist Cooperative Evolutionary Algorithm for Multi-Objective Reinforcement Learning

Abstract

Sequential decision-making problems with multiple objectives are known as multi-objective reinforcement learning. In these scenarios, decision-makers require a complete Pareto front that consists of Pareto optimal solutions. Such a front enables decision-makers to understand the relationship between objectives and make informed decisions from a broad range of solutions. However, existing methods may be unable to search for solutions in concave regions of the Pareto front or lack global optimization ability, leading to incomplete Pareto fronts. To address this issue, we propose an efficient elitist cooperative evolutionary algorithm that maintains both an evolving population and an elite archive. The elite archive uses cooperative operations with various genetic operators to guide the evolving population, resulting in efficient searches for Pareto optimal solutions. The experimental results on submarine treasure hunting benchmarks demonstrate the effectiveness of the proposed method in solving various multi-objective reinforcement learning problems and providing decision-makers with a set of trade-off solutions between travel time and treasure amount, enabling them to make flexible and informed decisions based on their preferences. Therefore, the proposed method has the potential to be a useful tool for implementing real-world applications.