Optimal model-free output synchronization of heterogeneous systems using off-policy reinforcement learning

H Modares, Subramanya Nageshrao, Gabriel Delgado Lopes, Robert Babuska, FL Lewis

Research output: Contribution to journalArticleScientificpeer-review

76 Citations (Scopus)

Abstract

This paper considers optimal output synchronization of heterogeneous linear multi-agent systems. Standard approaches to output synchronization of heterogeneous systems require either the solution of the output regulator equations or the incorporation of a p-copy of the leader’s dynamics in the controller of each agent. By contrast, in this paper neither one is needed. Moreover, here both the leader’s and the follower’s dynamics are assumed to be unknown. First, a distributed adaptive observer is designed to estimate the leader’s state for each agent. The output synchronization problem is then formulated as an optimal control problem and a novel model-free off-policy reinforcement learning algorithm is developed to solve the optimal output synchronization problem online in real time. It is shown that this optimal distributed approach implicitly solves the output regulation equations without actually doing so.
Simulation results are provided to verify the effectiveness of the proposed approach.
Original languageEnglish
Pages (from-to)334-341
JournalAutomatica
Volume71
DOIs
Publication statusPublished - 2016

Keywords

  • Output synchronization
  • Heterogeneous systems
  • Reinforcement learning
  • Leader–follower systems

Fingerprint

Dive into the research topics of 'Optimal model-free output synchronization of heterogeneous systems using off-policy reinforcement learning'. Together they form a unique fingerprint.

Cite this