Training and Transferring Safe Policies in Reinforcement Learning

Q. Yang*, T. D. Simão*, Nils Jansen, Simon H. Tindemans, M.T.J. Spaan

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

143 Downloads (Pure)

Abstract

Safety is critical to broadening the a lication of reinforcement learning (RL). Often, RL agents are trained in a controlled environment, such as a laboratory, before being de loyed in the real world. However, the target reward might be unknown rior to de loyment. Reward-free RL addresses this roblem by training an agent without the reward to ada t quickly once the reward is revealed.
We consider the constrained reward-free setting, where an agent (the guide) learns to ex lore safely without the reward signal. This agent is trained in a controlled environment, which allows unsafe interactions and still rovides the safety signal. After the target task is revealed, safety violations are not allowed anymore. Thus, the guide is leveraged to com ose a safe sam ling olicy. Drawing from transfer learning, we also regularize a target olicy (the student)
towards the guide while the student is unreliable and gradually eliminate the influence from the guide as training rogresses. The em irical analysis shows that this method can achieve safe transfer learning and hel s the student solve the target task faster.
Original languageEnglish
Title of host publicationProceedings of the Adaptive and Learning Agents Workshop
EditorsHayes Cruz , Santos da Silva
Number of pages14
Publication statusPublished - 2022
EventAdaptive and Learning Agents Workshop at AAMAS 2022 -
Duration: 9 May 202210 Jul 2022

Workshop

WorkshopAdaptive and Learning Agents Workshop at AAMAS 2022
Abbreviated titleALA 2022
Period9/05/2210/07/22

Fingerprint

Dive into the research topics of 'Training and Transferring Safe Policies in Reinforcement Learning'. Together they form a unique fingerprint.

Cite this