The use of reinforcement learning (RL) in real-world domains often requires extensive effort to ensure safe behavior. While this compromises the autonomy of the system, it might still be too risky to allow a learning agent to freely explore its environment. These strict impositions come at the cost of flexibility and applying them often relies on complex parameters and hard-coded knowledge modelled by the reward function. Autonomous driving is one such domain that could greatly benefit from more efficient and verifiable methods for safe automation. We propose to approach the automated driving problem using constrained RL, a method that automates the trade off between risk and utility, thereby significantly reducing the burden on the designer. We first show that an engineered reward function for ensuring safety and utility in one specific environment might not result in the optimal behavior when traffic dynamics changes in the exact environment. Next we show how algorithms based on constrained RL which are more robust to the environmental disturbances can address this challenge. These algorithms use a simple and easy to interpret reward and cost function, and are able to maintain both, efficiency and safety without requiring reward parameter tuning. We demonstrate our approach in the automated merging scenario with different traffic configurations such as low or high chance of cooperative drivers and different cooperative driving strategies.
|Title of host publication||Proceedings of the IEEE International Conference on Intelligent Transportation Systems|
|Number of pages||7|
|Publication status||Published - 2022|
|Event||2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC) - Macau, China|
Duration: 8 Oct 2022 → 12 Oct 2022
Conference number: 25th
|Name||IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC|
|Conference||2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC)|
|Period||8/10/22 → 12/10/22|
Bibliographical noteGreen Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.