Cooperative Learning of Multi-Agent Systems Via Reinforcement Learning

In many specific scenarios, accurateand practical cooperative learning is a commonly encountered challenge in multi-agent systems. Thus, the current investigation focuses on cooperative learning algorithms for multi-agent systems and underpins an alternate data-based neural network reinforcement learning framework. To achieve the data-based learning optimization, the proposed cooperative learning framework, which comprises two layers, introduces a virtual learning objective. The followers learn the behaviors of the virtual objects in the first layer based on the adaptive neural networks (NNs). Specifically, the actor and critic NNs are applied to acquire cooperative behaviors and assess this layer's long-term utility function. Then another layer realizes the tracking performance between the virtual objects and the leader by introducing the local data-based performance index. Then, we formulate a resulting deterministic optimization problem and resolve it effectively with the policy iteration algorithm. This intuitive cooperative learning algorithm also preserves good robustness properties and eliminates the dependence on the prior knowledge of the multi-agent system model in the solution process. Finally, a multi-robot formation system demonstrates this promising development's practical appeal and highly effective outcome.

Introduction

Cooperative learning of multi-agent systems facilitates an agent to perform objectives by interacting with its neighbor agents, which has encountered remarkable growth in past years and will continue to increase, given its capability of improving robustness and efficiency [1], [2], [3]. Cooperative learning plays a significant role in various fields, including intelligent transportation [4], aerospace systems [5], smart grids [6], etc. Reinforcement learning, as one of the most practical learning branches, has attained substantial attention in the multi-agent systems' cooperative learning community due to its online learning framework and simplicity of implementation [7], [8], [9]. Besides, reinforcement learning concerns how agents shall select actions in an environment such that some concepts of accumulative reward are maximized, and the environment can be formulated as a Markov decision process.