Abstract
We present an approach to safely reduce the communication required between agents in a Multi-Agent Reinforcement Learning system by exploiting the inherent robustness of the underlying Markov Decision Process. We compute robustness certificate functions (off-line), that give agents a conservative indication of how far their state measurements can deviate before they need to update other agents in the system with new measurements. This results in fully distributed decision functions, enabling agents to decide when it is necessary to communicate state variables. We derive bounds on the optimality of the resulting systems in terms of the discounted sum of rewards obtained, and show these bounds are a function of the design parameters. Additionally, we extend the results for the case where the robustness surrogate functions are learned from data, and present experimental results demonstrating a significant reduction in communication events between agents.
Original language | English |
---|---|
Title of host publication | Formal Modeling and Analysis of Timed Systems |
Subtitle of host publication | 20th International Conference, FORMATS 2022, Warsaw, Poland, September 13–15, 2022, Proceedings |
Editors | Sergiy Bogomolov, David Parker |
Publisher | Springer |
Pages | 281-297 |
ISBN (Electronic) | 978-3-031-15839-1 |
ISBN (Print) | 978-3-031-15838-4 |
DOIs | |
Publication status | Published - 2022 |
Event | 20th International Conference on Formal Modeling and Analysis of Timed Systems, FORMATS 2022 - Warsaw, Poland Duration: 13 Sept 2022 → 15 Sept 2022 |
Publication series
Name | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
---|---|
Volume | 13465 LNCS |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | 20th International Conference on Formal Modeling and Analysis of Timed Systems, FORMATS 2022 |
---|---|
Country/Territory | Poland |
City | Warsaw |
Period | 13/09/22 → 15/09/22 |
Bibliographical note
Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-careOtherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Keywords
- Event-Triggered Communication
- Multi-Agent Systems
- Reinforcement Learning