HCTA:Hierarchical Cooperative Task Allocation in Multi-Agent Reinforcement Learning

Abstract

Despite significant advances in multi-agent reinforcement learning (MARL) in recent years, learning about cooperative policies for complex tasks remains a challenge. The concept of task allocation provides a valuable tool for designing and comprehending intricate multi-agent systems, as it enables agents with similar roles to collaborate on the same subtasks. However, existing research efforts have not yet provided clear guidelines on how task allocation can better adapt to dynamic environments. This paper introduces a novel framework called Hierarchical Cooperative Task Allocation (HCTA) that is based on long-term behavior analysis. The proposed framework is designed to solve the dynamic task allocation problem in fully cooperative tasks within multi-agent systems. Firstly, we train a task selector based on action chains to enhance the efficiency of task allocation. Secondly, we introduce a hierarchical cooperative mechanism to realize the parallel training of upper-level task selection and lower-level task decision making, so as to accelerate the learning of cooperative strategy. Experimental results show that HTCA significantly improves the learning efficiency of joint policy through dynamic task allocation, and enhances the success rate in complex cooperative tasks in StarCraft II micromanagement benchmark.