Combined MPC and reinforcement learning for traffic signal control in urban traffic networks

Willemijn Remmerswaal, Dingshan Sun*, Anahita Jamshidnejad, Bart De Schutter

*Corresponding author for this work

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

Abstract

In general, the performance of model-based controllers cannot be guaranteed under model uncertainties or disturbances, while learning-based controllers require an extensively sufficient training process to perform well. These issues especially hold for large-scale nonlinear systems such as urban traffic networks. In this paper, a new framework is proposed by combining model predictive control (MPC) and reinforcement learning (RL) to provide desired performance for urban traffic networks even during the learning process, despite model uncertainties and disturbances. MPC and RL complement each other very well, since MPC provides a sub-optimal and constraint-satisfying control input while RL provides adaptive control laws and can handle uncertainties and disturbances. The resulting combined framework is applied for traffic signal control (TSC) of an urban traffic network. A case study is carried out to compare the performance of the proposed framework and other baseline controllers. Results show that the proposed combined framework outperforms conventional control methods under system uncertainties, in terms of reducing traffic congestion.

Original languageEnglish
Title of host publicationProceedings of the 26th International Conference on System Theory, Control and Computing, ICSTCC 2022
EditorsMarian Barbu, Razvan Solea
PublisherIEEE
Pages432-439
ISBN (Electronic)978-1-6654-6746-9
DOIs
Publication statusPublished - 2022
Event26th International Conference on System Theory, Control and Computing, ICSTCC 2022 - Sinaia, Romania
Duration: 19 Oct 202221 Oct 2022

Conference

Conference26th International Conference on System Theory, Control and Computing, ICSTCC 2022
Country/TerritoryRomania
CitySinaia
Period19/10/2221/10/22

Bibliographical note

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Fingerprint

Dive into the research topics of 'Combined MPC and reinforcement learning for traffic signal control in urban traffic networks'. Together they form a unique fingerprint.

Cite this