Abstract
Enabling the capability of assessing risk and making risk-aware decisions is essential to applying reinforcement learning to safety-critical robots like drones. In this paper, we investigate a specific case where a nano quadcopter robot learns to navigate an apriori-unknown cluttered environment under partial observability. We present a distributional reinforcement learning framework to generate adaptive risk-tendency policies. Specifically, we propose to use lower tail conditional variance of the learnt return distribution as intrinsic uncertainty estimation, and use exponentially weighted average forecasting (EWAF) to adapt the risk-tendency in accordance with the estimated uncertainty. In simulation and real-world empirical results, we show that (1) the most effective risk-tendency varies across states, (2) the agent with adaptive risk-tendency achieves superior performance compared to risk-neutral policy or risk-averse policy baselines. Code and video can be found in this repository: https://github.com/tudelft/risk-sensitive-rl.git
Original language | English |
---|---|
Title of host publication | Proceedings - ICRA 2023 |
Subtitle of host publication | IEEE International Conference on Robotics and Automation |
Publisher | IEEE |
Pages | 7198-7204 |
Number of pages | 7 |
ISBN (Electronic) | 9798350323658 |
DOIs | |
Publication status | Published - 2023 |
Event | 2023 IEEE International Conference on Robotics and Automation, ICRA 2023 - London, United Kingdom Duration: 29 May 2023 → 2 Jun 2023 |
Publication series
Name | Proceedings - IEEE International Conference on Robotics and Automation |
---|---|
Volume | 2023-May |
ISSN (Print) | 1050-4729 |
Conference
Conference | 2023 IEEE International Conference on Robotics and Automation, ICRA 2023 |
---|---|
Country/Territory | United Kingdom |
City | London |
Period | 29/05/23 → 2/06/23 |
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-careOtherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.