A Modern Perspective on Safe Automated Driving for Different Traffic Dynamics using Constrained Reinforcement Learning

Danial Kamran, Thiago D. Simão, Qisong Yang, Canmanie T. Ponnambalam, Johannes Fischer, Matthijs T.J. Spaan, Martin Lauer

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

The use of reinforcement learning (RL) in real-world domains often requires extensive effort to ensure safe behavior. While this compromises the autonomy of the system, it might still be too risky to allow a learning agent to freely explore its environment. These strict impositions come at the cost of flexibility and applying them often relies on complex parameters and hard-coded knowledge modelled by the reward function. Autonomous driving is one such domain that could greatly benefit from more efficient and verifiable methods for safe automation. We propose to approach the automated driving problem using constrained RL, a method that automates the trade off between risk and utility, thereby significantly reducing the burden on the designer. We first show that an engineered reward function for ensuring safety and utility in one specific environment might not result in the optimal behavior when traffic dynamics changes in the exact environment. Next we show how algorithms based on constrained RL which are more robust to the environmental disturbances can address this challenge. These algorithms use a simple and easy to interpret reward and cost function, and are able to maintain both, efficiency and safety without requiring reward parameter tuning. We demonstrate our approach in the automated merging scenario with different traffic configurations such as low or high chance of cooperative drivers and different cooperative driving strategies.
Original languageEnglish
Title of host publicationProceedings of the IEEE International Conference on Intelligent Transportation Systems
PublisherIEEE
Pages4017-4023
Number of pages7
ISBN (Electronic)978-1-6654-6880-0
ISBN (Print)978-1-6654-6881-7
DOIs
Publication statusPublished - 2022
Event2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC) - Macau, China
Duration: 8 Oct 202212 Oct 2022
Conference number: 25th

Publication series

NameIEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC
Volume2022-October

Conference

Conference2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)
Country/TerritoryChina
CityMacau
Period8/10/2212/10/22

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care

Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Fingerprint

Dive into the research topics of 'A Modern Perspective on Safe Automated Driving for Different Traffic Dynamics using Constrained Reinforcement Learning'. Together they form a unique fingerprint.

Cite this