Continuous control for high-dimensional state spaces: An interactive learning approach

Rodrigo Pérez-Dattari, Carlos Celemin, Javier Ruiz-Del-Solar, Jens Kober

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

10 Citations (Scopus)
36 Downloads (Pure)

Abstract

Deep Reinforcement Learning (DRL) has become a powerful methodology to solve complex decision-making problems. However, DRL has several limitations when used in real-world problems (e.g., robotics applications). For instance, long training times are required and cannot be accelerated in contrast to simulated environments, and reward functions may be hard to specify/model and/or to compute. Moreover, the transfer of policies learned in a simulator to the real-world has limitations (reality gap). On the other hand, machine learning methods that rely on the transfer of human knowledge to an agent have shown to be time efficient for obtaining well performing policies and do not require a reward function. In this context, we analyze the use of human corrective feedback during task execution to learn policies with high-dimensional state spaces, by using the D-COACH framework, and we propose new variants of this framework. D-COACH is a Deep Learning based extension of COACH (COrrective Advice Communicated by Humans), where humans are able to shape policies through corrective advice. The enhanced version of DCOACH, which is proposed in this paper, largely reduces the time and effort of a human for training a policy. Experimental results validate the efficiency of the D-COACH framework in three different problems (simulated and with real robots), and show that its enhanced version reduces the human training effort considerably, and makes it feasible to learn policies within periods of time in which a DRL agent do not reach any improvement.

Original languageEnglish
Title of host publicationProceedings of the International Conference on Robotics and Automation, ICRA 2019
Place of PublicationPiscataway, NJ, USA
PublisherIEEE
Pages7611-7617
ISBN (Electronic)978-1-5386-6026-3
DOIs
Publication statusPublished - 2019
Event2019 International Conference on Robotics and Automation, ICRA 2019 - Montreal, Canada
Duration: 20 May 201924 May 2019

Conference

Conference2019 International Conference on Robotics and Automation, ICRA 2019
Country/TerritoryCanada
CityMontreal
Period20/05/1924/05/19

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care

Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Fingerprint

Dive into the research topics of 'Continuous control for high-dimensional state spaces: An interactive learning approach'. Together they form a unique fingerprint.

Cite this