Structural integrity management via hierarchical resource allocation and continuous-control reinforcement learning

Research output: Contribution to conferencePaperpeer-review

27 Downloads (Pure)

Abstract

Maintenance planning of engineering systems is typically posed as a discrete stochastic optimal control problem, as it refers to determining a series of distinct interventions that upkeep structural integrity. Advanced algorithmic schemes within the joint framework of Partially Observable Markov Decision Processes (POMDPs) and multi-agent Deep Reinforcement Learning (DRL) have been recently able to approximate well global optima for this complex problem, outperforming existing time- and condition-based decision strategies. Integral to their success is the hypothesis that system components represent individual agents who form cooperative policies to minimize a central life-cycle cost. Thereby, the policy output scales linearly with the number of components, alleviating the curse of dimensionality related to combinatorial choices. State complexity and long-term optimality are handled efficiently via deep learning and POMDP principles, respectively. However, the efficiency of multi-agent coordination can fade as the number of agents increases. To this end, we propose a new formulation: we pose the problem as a continuous-control dynamic resource allocation one, combining hierarchical DRL and mixed-integer programming. Moving from flat decentralized to hierarchical multi-agent decompositions allows us to improve further the policy output scalability. The new Adaptive Knapsack Hierarchical Resource Allocator (AK-HRA) DRL architecture distributes available resources within the system, creating local, independently solvable, multi-choice knapsack optimization problems. By design, AK HRA allows decision-makers to inscribe known hierarchical structures and local decision rules in their architectures, thereby enhancing control and interpretability over the solution space. The efficacy of the new approach is demonstrated in a multi-component reliability system subject to stochastic deterioration.
Original languageEnglish
Number of pages8
Publication statusPublished - 2023
Event14th International Conference on Applications of Statistics and Probability in Civil Engineering 2023 - Trinity College Dublin, Dublin, Ireland
Duration: 9 Jul 202313 Jul 2023
https://icasp14.com/

Conference

Conference14th International Conference on Applications of Statistics and Probability in Civil Engineering 2023
Abbreviated titleICASP14
Country/TerritoryIreland
CityDublin
Period9/07/2313/07/23
Internet address

Fingerprint

Dive into the research topics of 'Structural integrity management via hierarchical resource allocation and continuous-control reinforcement learning'. Together they form a unique fingerprint.

Cite this