Abstraction-Guided Policy Recovery from Expert Demonstrations

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

25 Downloads (Pure)

Abstract

Behavior cloning is a method of automated decision-making that aims to extract meaningful information from expert demonstrations and reproduce the same behavior autonomously. It is unlikely that demonstrations will exhaustively cover the potential problem space, compromising the quality of automation when out-of-distribution states are encountered. Our approach RECO jointly learns both an imitation policy and recovery policy from expert data. The recovery policy steers the agent from unknown states back to the demonstrated states in the data set. While there is, per definition, no data available to learn the recovery policy, we exploit abstractions to generalize beyond the available data and simulate the recovery problem. When the most appropriate abstraction for the given data is unknown, our method selects the best recovery policy from a set generated by several candidate abstractions. In tabular domains, where we assume an agent must call to a human supervisor for help if it is in an unknown state, we show how RECO results in drastically fewer calls without compromising solution quality and with relatively few trajectories provided by an expert. We also introduce a continuous adaptation of our method and demonstrate the ability of RECO to recover an agent from states where its supervised learning-based imitation policy would otherwise fail.
Original languageEnglish
Title of host publication31th International Conference on Automated Planning and Scheduling
PublisherAmerican Association for Artificial Intelligence (AAAI)
Pages560-568
Number of pages9
Publication statusPublished - 2021
Event31st International Conference on Automated Planning and Scheduling - Virtual/online event
Duration: 7 Jun 202112 Jun 2021
Conference number: 31

Conference

Conference31st International Conference on Automated Planning and Scheduling
Abbreviated titleICAPS 2021
Period7/06/2112/06/21

Fingerprint

Dive into the research topics of 'Abstraction-Guided Policy Recovery from Expert Demonstrations'. Together they form a unique fingerprint.

Cite this