Deep reinforcement learning-based life-cycle management of deteriorating transportation systems

M. Saifullah, C.P. Andriotis, K.G. Papakonstantinou, S.M. Stoffels

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

4 Citations (Scopus)

Abstract

Efficient life-cycle bridge asset management delineates a planning optimization problem of paramount importance for the operational reliability of transportation infrastructure. It necessitates adept inspection and maintenance policies able to reduce risks and costs while incorporating long-term stochastic deterioration models, inference under uncertain structural health data, and various probabilistic and deterministic constraints. Structural integrity management policies for individual bridges, which are mere constituents of broader complex networks, cannot be devised in isolation of the policies of other system components, such as other bridges and pavement sections, and without considering system functions and traffic considerations. Such network effects render the optimization problem even harder to solve. Currently, age- or condition-based maintenance techniques, as well as risk-based or periodic inspection plans, have been used to address this class of challenging optimization problems. However, the efficacy of these techniques is often limited by optimality-, scalability-, and uncertainty-induced complexities. In practice, infrastructure management agencies often treat interconnected systems using disjoint plans for different component types, which in general do not ensure system-level optimality. To tackle the above, the optimization problem is herein cast within constrained Partially Observable Markov Decision Processes (POMDPs), which provide a comprehensive mathematical framework for stochastic sequential decision settings under observation/monitoring data uncertainty and limited resources. For the problem solution, the DDMAC algorithm (Deep Decentralized Multi-agent Actor-Critic) is successfully used, a deep reinforcement learning algorithm well-suited for management of large multi-state multi-component systems, as illustrated in an example application of an existing transportation network in Virginia, USA. The studied network comprises several bridge and pavement components exhibiting nonstationary deterioration, and various agency-imposed constraints, and traffic delay and risk factors are considered. Comparisons against conventional management policies showcase that the DDMAC solution significantly outperforms its counterparts.

Original languageEnglish
Title of host publicationBridge Safety, Maintenance, Management, Life-Cycle, Resilience and Sustainability
Subtitle of host publicationProceedings of the Eleventh International Conference on Bridge Maintenance, Safety and Management (IABMAS 2022), Barcelona, Spain, July 11-15, 2022
EditorsJ.R. Casas, D.M. Frangopol, J. Turmo
Place of PublicationLeiden
PublisherCRC Press / Balkema - Taylor & Francis Group
Pages293-301
Number of pages9
ISBN (Electronic)978-1-003-32264-1
ISBN (Print)978-1-032-34531-4, 978-1-032-34532-1
DOIs
Publication statusPublished - 2022
Event11th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2022 - Barcelona, Spain
Duration: 11 Jul 202215 Jul 2022

Conference

Conference11th International Conference on Bridge Maintenance, Safety and Management, IABMAS 2022
Country/TerritorySpain
CityBarcelona
Period11/07/2215/07/22

Bibliographical note

The authors acknowledge the support of the U.S. National Science Foundation under CAREER Grant No. 1751941 and LEAP-HI Grant No. 2053620, and the Center for Integrated Asset Management for Multimodal Transportation Infrastructure Systems, 2018 U.S. DOT Region 3 University Center. Dr. Andriotis would further like to acknowledge the support of the TU Delft AI Labs program.

Fingerprint

Dive into the research topics of 'Deep reinforcement learning-based life-cycle management of deteriorating transportation systems'. Together they form a unique fingerprint.

Cite this