Reinforcement Learning by Guided Safe Exploration

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

23 Downloads (Pure)

Abstract

Safety is critical to broadening the application of reinforcement learning (RL). Often, we train RL agents in a controlled environment, such as a laboratory, before deploying them in the real world. However, the real-world target task might be unknown prior to deployment. Reward-free RL trains an agent without the reward to adapt quickly once the reward is revealed. We consider the constrained reward-free setting, where an agent (the guide) learns to explore safely without the reward signal. This agent is trained in a controlled environment, which allows unsafe interactions and still provides the safety signal. After the target task is revealed, safety violations are not allowed anymore. Thus, the guide is leveraged to compose a safe behaviour policy. Drawing from transfer learning, we also regularize a target policy (the student) towards the guide while the student is unreliable and gradually eliminate the influence of the guide as training progresses. The empirical analysis shows that this method can achieve safe transfer learning and helps the student solve the target task faster.
Original languageEnglish
Title of host publicationECAI 2023 - 26th European Conference on Artificial Intelligence, including 12th Conference on Prestigious Applications of Intelligent Systems, PAIS 2023 - Proceedings
EditorsKobi Gal, Kobi Gal, Ann Nowe, Grzegorz J. Nalepa, Roy Fairstein, Roxana Radulescu
Pages2858 - 2865
Number of pages8
ISBN (Electronic)9781643684369
DOIs
Publication statusPublished - 2023
Event26th European Conference on Artificial Intelligence - Kraków, Poland
Duration: 30 Sept 20234 Oct 2023
Conference number: 26

Publication series

NameFrontiers in Artificial Intelligence and Applications
Volume372
ISSN (Print)0922-6389
ISSN (Electronic)1879-8314

Conference

Conference26th European Conference on Artificial Intelligence
Abbreviated titleECAI 2023
Country/TerritoryPoland
CityKraków
Period30/09/234/10/23

Fingerprint

Dive into the research topics of 'Reinforcement Learning by Guided Safe Exploration'. Together they form a unique fingerprint.

Cite this