Effects of Shaping a Reward on Multiagent Reinforcement Learning

Abstract

The multiagent reinforcement learning approach is now widely applied to cause agents to behave rationally in a multiagent system. However, due to the complex interactions in a multiagent domain, it is difficult to decide the each agent’s fair share of the reward for contributing to the goal achievement. This chapter reviews a reward shaping problem that defines when and what amount of reward should be given to agents. We employ keepaway soccer as a typical multiagent continuing task that requires skilled collaboration between the agents. Shaping the reward structure for this domain is difficult for the following reasons: i) a continuing task such as keepaway soccer has no explicit goal, and so it is hard to determine when a reward should be given to the agents, ii) in such a multiagent cooperative task, it is difficult to fairly share the reward for each agent‘s contribution. Through experiments, we found that reward shaping has a major effect on an agent‘s behavior.