Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning

Nick van den Bovenkamp, Juan S. Giraldo, Edgar Mauricio Salazar Duque, Pedro P. Vergara , Charalambos Konstantinou, Peter Palensky

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review


This paper introduces an energy management system (EMS) aiming to minimize electricity operating costs using reinforcement learning (RL) with a linear function approximation. The proposed EMS uses a Q-learning with tile coding (QLTC) algorithm and is compared to a deterministic mixed-integer linear programming (MILP) with perfect forecast information. The comparison is performed using a case study on an industrial manufacturing company in the Netherlands, considering measured electricity consumption, PV generation, and wholesale electricity prices during one week of operation. The results show that the proposed EMS can adjust the prosumer's power consumption considering favorable prices. The electricity costs obtained using the QLTC algorithm are 99% close to those obtained with the MILP model. Furthermore, the results demonstrate that the QLTC model can generalize on previously learned control policies even in the case of missing data and can deploy actions 80% near to the MILP's optimal solution.
Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE Belgrade PowerTech
Place of PublicationPiscataway
Number of pages6
ISBN (Electronic)978-1-6654-8778-8
ISBN (Print)978-1-6654-8779-5
Publication statusPublished - 2023
Event2023 IEEE Belgrade PowerTech - Belgrade, Serbia
Duration: 25 Jun 202329 Jun 2023

Publication series

Name2023 IEEE Belgrade PowerTech, PowerTech 2023


Conference2023 IEEE Belgrade PowerTech

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • Q-learning
  • tile coding
  • energy management system
  • mixed-integer linear programming


Dive into the research topics of 'Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning'. Together they form a unique fingerprint.

Cite this