Secondary Frequency Control of Microgrids: An Online Reinforcement Learning Approach

Mahya Adibi*, Jacob Van Der Woude

*Corresponding author for this work

Research output: Contribution to journalArticleScientificpeer-review

10 Citations (Scopus)
2 Downloads (Pure)


In this article, we present a reinforcement learning-based scheme for secondary frequency control of lossy inverter-based microgrids. Compared with the existing methods in the literature, we relax the common restrictions on the system, i.e., being lossless, and the transmission lines and loads to have known constant impedances. The proposed secondary frequency control scheme does not require a priori information about system parameters and can achieve frequency synchronization within an ultimate bound in the presence of dominantly resistive and/or inductive line and load impedances, model parameter uncertainties, and time varying loads and disturbances. First, using Lyapunov theory, a feedback control is formulated based on the unknown dynamics of the microgrid. Next, a performance function is defined based on cumulative costs toward achieving convergence to the nominal frequency. The performance function is approximated by a critic neural network in real-time. An actor network is then simultaneously learning a parameterized approximation of the nonlinear dynamics and optimizing the approximated performance function obtained from the critic network. Furthermore, using the Lyapunov approach, the uniformly ultimate boundedness of the closed-loop frequency error dynamics and the networks' weight estimation errors are shown.

Original languageEnglish
Pages (from-to)4824-4831
Number of pages8
JournalIEEE Transactions on Automatic Control
Issue number9
Publication statusPublished - 2022

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • Microgrids
  • neural network
  • reinforcement learning
  • secondary frequency control

Cite this