Influence-Based Abstraction in Deep Reinforcement Learning

Miguel Suau de Castro; Elena Congeduti; Rolf Starre; Aleksander Czechowski; Frans Oliehoek

Influence-Based Abstraction in Deep Reinforcement Learning

Miguel Suau de Castro, Elena Congeduti, Rolf Starre, Aleksander Czechowski, Frans Oliehoek

Interactive Intelligence

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

75 Downloads (Pure)

Abstract

thousands, or even millions of state variables. Unfortunately, applying reinforcement learning algorithms to handle complex tasks becomes more and more challenging as the number of state variables increases. In this paper, we build on the concept of influence-based abstraction which tries to tackle such scalability issues by decomposing large systems into small regions. We explore this method in the context of deep reinforcement learning, showing that by keeping track of a small set of variables in the history of previous actions and observations we can learn policies that can effectively control a local region in the global system.

Original language	English
Title of host publication	AAMAS Workshop on Adaptive Learning Agents (ALA)
Place of Publication	Montreal, Canada
Number of pages	8
Publication status	Published - 13 May 2019
Event	ALA 2019 - Workshop at AAMAS 2019 - Montreal, Canada Duration: 13 May 2019 → 14 May 2019

Workshop

Workshop	ALA 2019 - Workshop at AAMAS 2019
Country/Territory	Canada
City	Montreal
Period	13/05/19 → 14/05/19

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Reinforcement Learning
Dec-POMDP
Influence-based abstraction

Access to Document

Suau19ALAFinal published version, 925 KBLicence: Unspecified

https://ala2019.vub.ac.be/

Cite this

@inproceedings{e39070311af345838741b6b4ed729170,

title = "Influence-Based Abstraction in Deep Reinforcement Learning",

abstract = "thousands, or even millions of state variables. Unfortunately, applying reinforcement learning algorithms to handle complex tasks becomes more and more challenging as the number of state variables increases. In this paper, we build on the concept of influence-based abstraction which tries to tackle such scalability issues by decomposing large systems into small regions. We explore this method in the context of deep reinforcement learning, showing that by keeping track of a small set of variables in the history of previous actions and observations we can learn policies that can effectively control a local region in the global system. ",

keywords = "Reinforcement Learning, Dec-POMDP, Influence-based abstraction",

author = "{Suau de Castro}, Miguel and Elena Congeduti and Rolf Starre and Aleksander Czechowski and Frans Oliehoek",

note = "Green Open Access added to TU Delft Institutional Repository {\textquoteleft}You share, we take care!{\textquoteright} – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; ALA 2019 - Workshop at AAMAS 2019 ; Conference date: 13-05-2019 Through 14-05-2019",

year = "2019",

month = may,

day = "13",

language = "English",

booktitle = "AAMAS Workshop on Adaptive Learning Agents (ALA)",

}

TY - GEN

T1 - Influence-Based Abstraction in Deep Reinforcement Learning

AU - Suau de Castro, Miguel

AU - Congeduti, Elena

AU - Starre, Rolf

AU - Czechowski, Aleksander

AU - Oliehoek, Frans

N1 - Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2019/5/13

Y1 - 2019/5/13

N2 - thousands, or even millions of state variables. Unfortunately, applying reinforcement learning algorithms to handle complex tasks becomes more and more challenging as the number of state variables increases. In this paper, we build on the concept of influence-based abstraction which tries to tackle such scalability issues by decomposing large systems into small regions. We explore this method in the context of deep reinforcement learning, showing that by keeping track of a small set of variables in the history of previous actions and observations we can learn policies that can effectively control a local region in the global system.

AB - thousands, or even millions of state variables. Unfortunately, applying reinforcement learning algorithms to handle complex tasks becomes more and more challenging as the number of state variables increases. In this paper, we build on the concept of influence-based abstraction which tries to tackle such scalability issues by decomposing large systems into small regions. We explore this method in the context of deep reinforcement learning, showing that by keeping track of a small set of variables in the history of previous actions and observations we can learn policies that can effectively control a local region in the global system.

KW - Reinforcement Learning

KW - Dec-POMDP

KW - Influence-based abstraction

M3 - Conference contribution

BT - AAMAS Workshop on Adaptive Learning Agents (ALA)

CY - Montreal, Canada

T2 - ALA 2019 - Workshop at AAMAS 2019

Y2 - 13 May 2019 through 14 May 2019

ER -

Influence-Based Abstraction in Deep Reinforcement Learning

Abstract

Workshop

Bibliographical note

Keywords

Access to Document

Fingerprint

Cite this