Decentralized Reinforcement Learning of robot behaviors

David L. Leottau*, Javier Ruiz-del-Solar, Robert Babuška

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

41 Citations (Scopus)
53 Downloads (Pure)

Abstract

A multi-agent methodology is proposed for Decentralized Reinforcement Learning (DRL) of individual behaviors in problems where multi-dimensional action spaces are involved. When using this methodology, sub-tasks are learned in parallel by individual agents working toward a common goal. In addition to proposing this methodology, three specific multi agent DRL approaches are considered: DRL-Independent, DRL Cooperative-Adaptive (CA), and DRL-Lenient. These approaches are validated and analyzed with an extensive empirical study using four different problems: 3D Mountain Car, SCARA Real-Time Trajectory Generation, Ball-Dribbling in humanoid soccer robotics, and Ball-Pushing using differential drive robots. The experimental validation provides evidence that DRL implementations show better performances and faster learning times than their centralized counterparts, while using less computational resources. DRL-Lenient and DRL-CA algorithms achieve the best final performances for the four tested problems, outperforming their DRL-Independent counterparts. Furthermore, the benefits of the DRL-Lenient and DRL-CA are more noticeable when the problem complexity increases and the centralized scheme becomes intractable given the available computational resources and training time.

Original languageEnglish
Pages (from-to)130-159
JournalArtificial Intelligence
Volume256
DOIs
Publication statusPublished - 2018

Bibliographical note

Accepted Author Manuscript

Keywords

  • Autonomous robots
  • Decentralized control
  • Distributed artificial intelligence
  • Multi-agent systems
  • Reinforcement learning

Fingerprint

Dive into the research topics of 'Decentralized Reinforcement Learning of robot behaviors'. Together they form a unique fingerprint.

Cite this