The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning

Jacopo Castellini; Frans A. Oliehoek; Rahul Savani; Shimon Whiteson

The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning

Jacopo Castellini, Frans A. Oliehoek, Rahul Savani, Shimon Whiteson

Interactive Intelligence

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

22 Downloads (Pure)

Abstract

Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. In this work, we empirically investigate the representational power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results quantify how well various approaches can represent the requisite value functions, and help us identify issues that can impede good performance.

Original language	English
Title of host publication	AAMAS'19
Subtitle of host publication	Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS)
Publisher	International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)
Pages	1862-1864
Number of pages	3
ISBN (Print)	978-1-4503-6309-9
Publication status	Published - 2019
Event	AAMAS 2019: The 18th International Conference on Autonomous Agents and MultiAgent Systems - Montreal, Canada Duration: 13 May 2019 → 17 May 2019

Conference

Conference	AAMAS 2019
Country	Canada
City	Montreal
Period	13/05/19 → 17/05/19

Keywords

multi-agent systems
neural networks
decision-making
actionvaluerepresentation
one-shotgames

Access to Document

TRCFinal published version, 1.11 MB

https://dl.acm.org/citation.cfm?id=3331944

Fingerprint Dive into the research topics of 'The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning'. Together they form a unique fingerprint.

Cite this

Castellini, J., Oliehoek, F. A., Savani, R., & Whiteson, S. (2019). The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. In AAMAS'19: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (pp. 1862-1864 ). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). https://dl.acm.org/citation.cfm?id=3331944

Castellini, Jacopo ; Oliehoek, Frans A. ; Savani, Rahul ; Whiteson, Shimon. / The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. AAMAS'19: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2019. pp. 1862-1864

@inproceedings{8ca4ef096fc847deb2c9bc33671854ca,

title = "The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning",

abstract = "Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. In this work, we empirically investigate the representational power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results quantify how well various approaches can represent the requisite value functions, and help us identify issues that can impede good performance.",

keywords = "multi-agent systems, neural networks, decision-making, actionvaluerepresentation, one-shotgames",

author = "Jacopo Castellini and Oliehoek, {Frans A.} and Rahul Savani and Shimon Whiteson",

note = "Green Open Access added to TU Delft Institutional Repository {\textquoteleft}You share, we take care!{\textquoteright} – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. ; AAMAS 2019 : The 18th International Conference on Autonomous Agents and MultiAgent Systems ; Conference date: 13-05-2019 Through 17-05-2019",

year = "2019",

language = "English",

isbn = "978-1-4503-6309-9",

pages = "1862--1864 ",

booktitle = "AAMAS'19",

publisher = "International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)",

}

Castellini, J, Oliehoek, FA, Savani, R & Whiteson, S 2019, The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. in AAMAS'19: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), pp. 1862-1864 , AAMAS 2019, Montreal, Canada, 13/05/19. <https://dl.acm.org/citation.cfm?id=3331944>

The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning. / Castellini, Jacopo; Oliehoek, Frans A.; Savani, Rahul; Whiteson, Shimon.

AAMAS'19: Proceedings of the Eighteenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS). International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS), 2019. p. 1862-1864 .

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - The Representational Capacity of Action-Value Networks for Multi-Agent Reinforcement Learning

AU - Castellini, Jacopo

AU - Oliehoek, Frans A.

AU - Savani, Rahul

AU - Whiteson, Shimon

N1 - Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2019

Y1 - 2019

N2 - Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. In this work, we empirically investigate the representational power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results quantify how well various approaches can represent the requisite value functions, and help us identify issues that can impede good performance.

AB - Recent years have seen the application of deep reinforcement learning techniques to cooperative multi-agent systems, with great empirical success. In this work, we empirically investigate the representational power of various network architectures on a series of one-shot games. Despite their simplicity, these games capture many of the crucial problems that arise in the multi-agent setting, such as an exponential number of joint actions or the lack of an explicit coordination mechanism. Our results quantify how well various approaches can represent the requisite value functions, and help us identify issues that can impede good performance.

KW - multi-agent systems

KW - neural networks

KW - decision-making

KW - actionvaluerepresentation

KW - one-shotgames

M3 - Conference contribution

SN - 978-1-4503-6309-9

SP - 1862

EP - 1864

BT - AAMAS'19

PB - International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)

T2 - AAMAS 2019

Y2 - 13 May 2019 through 17 May 2019

ER -