Human demonstrations for fast and safe exploration in reinforcement learning

Gerben Schonebaum, Jaime Junell, Erik Jan van Kampen

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

26 Downloads (Pure)

Abstract

Reinforcement learning is a promising framework for controlling complex vehicles with a high level of autonomy, since it does not need a dynamic model of the vehicle, and it is able to adapt to changing conditions. When learning from scratch, the performance of a reinforcement learning controller may initially be poor and -for real life applications- unsafe. In this paper the effects of using human demonstrations on the performance of reinforcement learning is investigated, using a combination of offline and online least squares policy iteration. It is found that using the human as an efficient explorer improves learning time and performance for a benchmark reinforcement learning problem. The benefit of the human demonstration is larger for problems where the human can make use of its understanding of the problem to efficiently explore the state space. Applied to a simplified quadrotor slung load drop off problem, the use of human demonstrations reduces the number of crashes during learning. As such, this paper contributes to safer and faster learning for model-free, adaptive control problems.

Original languageEnglish
Title of host publicationAIAA Information Systems-AIAA Infotech at Aerospace, 2017
PublisherAmerican Institute of Aeronautics and Astronautics Inc. (AIAA)
Number of pages18
ISBN (Electronic)9781624104497
DOIs
Publication statusPublished - 2017
EventAIAA Information Systems-Infotech At Aerospace Conference, 2017 - Grapevine, United States
Duration: 9 Jan 201713 Jan 2017

Conference

ConferenceAIAA Information Systems-Infotech At Aerospace Conference, 2017
CountryUnited States
CityGrapevine
Period9/01/1713/01/17

Fingerprint Dive into the research topics of 'Human demonstrations for fast and safe exploration in reinforcement learning'. Together they form a unique fingerprint.

Cite this