The role of value of information in multi-agent deep reinforcement learning for optimal decision-making under uncertainty

Mohammad Saifullah, Charalampos Andriotis, Konstantinos G. Papakonstantinou

Research output: Contribution to conferencePaperpeer-review

13 Downloads (Pure)

Abstract

To preserve structural safety of deteriorating engineering systems through optimal maintenance, it is imperative to efficiently integrate structural health information with decision-making optimization frameworks. Although there may be abundance of available data, these are often uncertain and incomplete. In addition, joint inspection and maintenance (I&M) optimization is inherently complex due to high-dimensional state and action spaces, stochastic objectives, long planning horizons, and various constraints, among others. As shown recently, these computational challenges can be effectively addressed through optimization principles of Partially Observable Markov Decision Processes (POMDPs) and constrained Deep Reinforcement Learning (DRL). The POMDP framework provides a way of updating the decision-maker's perception about the system state by naturally incorporating the Value of Information (VoI) in the optimality equations. As such, optimal observation-gathering actions are those which guide maintenance decisions towards reduced life-cycle costs and risks. The role of VoI in DRL-driven I&M has also been shown to be central to the formation of policy gradients, which are necessary to obtain the optimal I&M plan with deep learning actor-critic architectures. Leveraging this property, a recently devised DRL architecture is further examined in this work, consisting of fully decoupled 'maintainer' and 'inspector' actors, which allow for greater efficacy and interpretability in multi-agent DRL settings. Several numerical analyses are carried out to assess the performance of the relevant architectures on stochastic systems with a varying number of components, multiple maintenance-inspection actions per component, and system-level failure risks.
Original languageEnglish
Number of pages8
Publication statusPublished - 2023
Event14th International Conference on Applications of Statistics and Probability in Civil Engineering 2023 - Trinity College Dublin, Dublin, Ireland
Duration: 9 Jul 202313 Jul 2023
https://icasp14.com/

Conference

Conference14th International Conference on Applications of Statistics and Probability in Civil Engineering 2023
Abbreviated titleICASP14
Country/TerritoryIreland
CityDublin
Period9/07/2313/07/23
Internet address

Fingerprint

Dive into the research topics of 'The role of value of information in multi-agent deep reinforcement learning for optimal decision-making under uncertainty'. Together they form a unique fingerprint.

Cite this