Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

C. P. Andriotis*, K. G. Papakonstantinou

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

20 Citations (Scopus)
13 Downloads (Pure)


Determination of inspection and maintenance policies for minimizing long-term risks and costs in deteriorating engineering environments constitutes a complex optimization problem. Major computational challenges include the (i) curse of dimensionality, due to exponential scaling of state/action set cardinalities with the number of components; (ii) curse of history, related to exponentially growing decision-trees with the number of decision-steps; (iii) presence of state uncertainties, induced by inherent environment stochasticity and variability of inspection/monitoring measurements; (iv) presence of constraints, pertaining to stochastic long-term limitations, due to resource scarcity and other infeasible/undesirable system responses. In this work, these challenges are addressed within a joint framework of constrained Partially Observable Markov Decision Processes (POMDP) and multi-agent Deep Reinforcement Learning (DRL). POMDPs optimally tackle (ii)-(iii), combining stochastic dynamic programming with Bayesian inference principles. Multi-agent DRL addresses (i), through deep function parametrizations and decentralized control assumptions. Challenge (iv) is herein handled through proper state augmentation and Lagrangian relaxation, with emphasis on life-cycle risk-based constraints and budget limitations. The underlying algorithmic steps are provided, and the proposed framework is found to outperform well-established policy baselines and facilitate adept prescription of inspection and intervention actions, in cases where decisions must be made in the most resource- and risk-aware manner.

Original languageEnglish
Article number107551
Number of pages16
JournalReliability Engineering and System Safety
Publication statusPublished - 2021

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project

Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • Constrained stochastic optimization
  • Decentralized multi-agent control
  • Deep reinforcement learning
  • Inspection and maintenance planning
  • Partially observable Markov decision processes
  • System risk and reliability


Dive into the research topics of 'Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints'. Together they form a unique fingerprint.

Cite this