The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning

Jacopo Castellini; Frans A. Oliehoek; Rahul Savani; Shimon Whiteson

The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning

Jacopo Castellini, Frans A. Oliehoek, Rahul Savani, Shimon Whiteson

Interactive Intelligence

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

13 Citations (Scopus)

27 Downloads (Pure)

Abstract

Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. In this work, we empirically investigate the representational power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results quantify how well various approaches can represent the requisite value functions, and help us identify issues that can impede good performance.

Original language	English
Title of host publication	18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019
Subtitle of host publication	Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
Publisher	International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Pages	1862-1864
Number of pages	3
ISBN (Electronic)	9781510892002
ISBN (Print)	978-1-4503-6309-9
Publication status	Published - 2019
Event	AAMAS 2019: The 18th International Conference on Autonomous Agents and MultiAgent Systems - Montreal, Canada Duration: 13 May 2019 → 17 May 2019

Publication series

Name	Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS
Volume	4
ISSN (Print)	1548-8403
ISSN (Electronic)	1558-2914

Conference

Conference	AAMAS 2019
Country/Territory	Canada
City	Montreal
Period	13/05/19 → 17/05/19

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care

Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Actionvalue representation
Decision-making
Multi-agent systems
Neural networks
One-shot games

Access to Document

TRCFinal published version, 1.11 MB

https://dl.acm.org/citation.cfm?id=3331944

Cite this

Castellini, J., Oliehoek, F. A., Savani, R., & Whiteson, S. (2019). The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. In 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (pp. 1862-1864 ). (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS; Vol. 4). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). https://dl.acm.org/citation.cfm?id=3331944

Castellini, Jacopo ; Oliehoek, Frans A. ; Savani, Rahul ; Whiteson, Shimon. / The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2019. pp. 1862-1864 (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS).

@inproceedings{8ca4ef096fc847deb2c9bc33671854ca,

title = "The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning",

abstract = "Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. In this work, we empirically investigate the representational power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results quantify how well various approaches can represent the requisite value functions, and help us identify issues that can impede good performance.",

keywords = "Actionvalue representation, Decision-making, Multi-agent systems, Neural networks, One-shot games",

author = "Jacopo Castellini and Oliehoek, {Frans A.} and Rahul Savani and Shimon Whiteson",

note = "Green Open Access added to TU Delft Institutional Repository {\textquoteleft}You share, we take care!{\textquoteright} – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. ; AAMAS 2019 : The 18th International Conference on Autonomous Agents and MultiAgent Systems ; Conference date: 13-05-2019 Through 17-05-2019",

year = "2019",

language = "English",

isbn = "978-1-4503-6309-9",

series = "Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS",

publisher = "International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)",

pages = "1862--1864 ",

booktitle = "18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019",

}

Castellini, J, Oliehoek, FA, Savani, R & Whiteson, S 2019, The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. in 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, vol. 4, International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), pp. 1862-1864 , AAMAS 2019, Montreal, Canada, 13/05/19. <https://dl.acm.org/citation.cfm?id=3331944>

The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. / Castellini, Jacopo; Oliehoek, Frans A.; Savani, Rahul; Whiteson, Shimon.

18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2019. p. 1862-1864 (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS; Vol. 4).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning

AU - Castellini, Jacopo

AU - Oliehoek, Frans A.

AU - Savani, Rahul

AU - Whiteson, Shimon

N1 - Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2019

Y1 - 2019

N2 - Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. In this work, we empirically investigate the representational power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results quantify how well various approaches can represent the requisite value functions, and help us identify issues that can impede good performance.

AB - Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. In this work, we empirically investigate the representational power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results quantify how well various approaches can represent the requisite value functions, and help us identify issues that can impede good performance.

KW - Actionvalue representation

KW - Decision-making

KW - Multi-agent systems

KW - Neural networks

KW - One-shot games

UR - http://www.scopus.com/inward/record.url?scp=85077018268&partnerID=8YFLogxK

M3 - Conference contribution

SN - 978-1-4503-6309-9

T3 - Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS

SP - 1862

EP - 1864

BT - 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019

PB - International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)

T2 - AAMAS 2019

Y2 - 13 May 2019 through 17 May 2019

ER -

Castellini J, Oliehoek FA, Savani R, Whiteson S. The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. In 18th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2019: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). 2019. p. 1862-1864 . (Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS).

The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this