DeepKoCo: Efficient latent planning with a task-relevant Koopman representation

Bas van der Heijden; Laura Ferranti; Jens Kober; Robert Babuska

doi:10.1109/IROS51168.2021.9636408

DeepKoCo: Efficient latent planning with a task-relevant Koopman representation

Bas van der Heijden, Laura Ferranti, Jens Kober, Robert Babuska

Learning & Autonomous Control

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

1 Citation (Scopus)

13 Downloads (Pure)

Abstract

This paper presents DeepKoCo, a novel modelbased agent that learns a latent Koopman representation from images. This representation allows DeepKoCo to plan efficiently using linear control methods, such as linear model predictive control. Compared to traditional agents, DeepKoCo learns taskrelevant dynamics, thanks to the use of a tailored lossy autoencoder network that allows DeepKoCo to learn latent dynamics that reconstruct and predict only observed costs, rather than all observed dynamics. As our results show, DeepKoCo achieves a similar final performance as traditional model-free methods on complex control tasks, while being considerably more robust to distractor dynamics, making the proposed agent more amenable for real-life applications.

Original language	English
Title of host publication	Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021)
Publisher	IEEE
Pages	183-189
ISBN (Electronic)	978-1-6654-1714-3
ISBN (Print)	978-1-6654-1715-0
DOIs	https://doi.org/10.1109/IROS51168.2021.9636408
Publication status	Published - 2021
Event	2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) - Online at Prague, Czech Republic Duration: 27 Sep 2021 → 1 Oct 2021

Conference

Conference	2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Country/Territory	Czech Republic
City	Online at Prague
Period	27/09/21 → 1/10/21

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Model-based reinforcement learning
Koopman theory
model-predictive control

Access to Document

10.1109/IROS51168.2021.9636408

DeepKoCo_Efficient_latent_planning_with_a_task-relevant_Koopman_representationFinal published version, 1.86 MB

Cite this

@inproceedings{cc306fd6b16a4756b2bac2fb2de5290c,

title = "DeepKoCo: Efficient latent planning with a task-relevant Koopman representation",

abstract = "This paper presents DeepKoCo, a novel modelbased agent that learns a latent Koopman representation from images. This representation allows DeepKoCo to plan efficiently using linear control methods, such as linear model predictive control. Compared to traditional agents, DeepKoCo learns taskrelevant dynamics, thanks to the use of a tailored lossy autoencoder network that allows DeepKoCo to learn latent dynamics that reconstruct and predict only observed costs, rather than all observed dynamics. As our results show, DeepKoCo achieves a similar final performance as traditional model-free methods on complex control tasks, while being considerably more robust to distractor dynamics, making the proposed agent more amenable for real-life applications.",

keywords = "Model-based reinforcement learning, Koopman theory, model-predictive control",

author = "{van der Heijden}, Bas and Laura Ferranti and Jens Kober and Robert Babuska",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) ; Conference date: 27-09-2021 Through 01-10-2021",

year = "2021",

doi = "10.1109/IROS51168.2021.9636408",

language = "English",

isbn = "978-1-6654-1715-0",

pages = "183--189",

booktitle = "Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021)",

publisher = "IEEE ",

address = "United States",

}

van der Heijden, B , Ferranti, L , Kober, J & Babuska, R 2021, DeepKoCo: Efficient latent planning with a task-relevant Koopman representation. in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021). IEEE , pp. 183-189, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Online at Prague, Czech Republic, 27/09/21. https://doi.org/10.1109/IROS51168.2021.9636408

DeepKoCo : Efficient latent planning with a task-relevant Koopman representation. / van der Heijden, Bas ; Ferranti, Laura ; Kober, Jens ; Babuska, Robert.

Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021). IEEE , 2021. p. 183-189.

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - DeepKoCo

T2 - 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

AU - van der Heijden, Bas

AU - Ferranti, Laura

AU - Kober, Jens

AU - Babuska, Robert

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2021

Y1 - 2021

N2 - This paper presents DeepKoCo, a novel modelbased agent that learns a latent Koopman representation from images. This representation allows DeepKoCo to plan efficiently using linear control methods, such as linear model predictive control. Compared to traditional agents, DeepKoCo learns taskrelevant dynamics, thanks to the use of a tailored lossy autoencoder network that allows DeepKoCo to learn latent dynamics that reconstruct and predict only observed costs, rather than all observed dynamics. As our results show, DeepKoCo achieves a similar final performance as traditional model-free methods on complex control tasks, while being considerably more robust to distractor dynamics, making the proposed agent more amenable for real-life applications.

AB - This paper presents DeepKoCo, a novel modelbased agent that learns a latent Koopman representation from images. This representation allows DeepKoCo to plan efficiently using linear control methods, such as linear model predictive control. Compared to traditional agents, DeepKoCo learns taskrelevant dynamics, thanks to the use of a tailored lossy autoencoder network that allows DeepKoCo to learn latent dynamics that reconstruct and predict only observed costs, rather than all observed dynamics. As our results show, DeepKoCo achieves a similar final performance as traditional model-free methods on complex control tasks, while being considerably more robust to distractor dynamics, making the proposed agent more amenable for real-life applications.

KW - Model-based reinforcement learning

KW - Koopman theory

KW - model-predictive control

UR - http://www.scopus.com/inward/record.url?scp=85124360777&partnerID=8YFLogxK

U2 - 10.1109/IROS51168.2021.9636408

DO - 10.1109/IROS51168.2021.9636408

M3 - Conference contribution

SN - 978-1-6654-1715-0

SP - 183

EP - 189

BT - Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021)

PB - IEEE

Y2 - 27 September 2021 through 1 October 2021

ER -

DeepKoCo: Efficient latent planning with a task-relevant Koopman representation

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this