Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning

Nick  van den Bovenkamp; Juan S. Giraldo; Edgar Mauricio Salazar Duque; Pedro P. Vergara; Charalambos  Konstantinou; Peter Palensky

doi:10.1109/PowerTech55446.2023.10202699

Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning

Nick van den Bovenkamp, Juan S. Giraldo, Edgar Mauricio Salazar Duque, Pedro P. Vergara , Charalambos Konstantinou, Peter Palensky

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

22 Downloads (Pure)

Abstract

This paper introduces an energy management system (EMS) aiming to minimize electricity operating costs using reinforcement learning (RL) with a linear function approximation. The proposed EMS uses a Q-learning with tile coding (QLTC) algorithm and is compared to a deterministic mixed-integer linear programming (MILP) with perfect forecast information. The comparison is performed using a case study on an industrial manufacturing company in the Netherlands, considering measured electricity consumption, PV generation, and wholesale electricity prices during one week of operation. The results show that the proposed EMS can adjust the prosumer's power consumption considering favorable prices. The electricity costs obtained using the QLTC algorithm are 99% close to those obtained with the MILP model. Furthermore, the results demonstrate that the QLTC model can generalize on previously learned control policies even in the case of missing data and can deploy actions 80% near to the MILP's optimal solution.

Original language	English
Title of host publication	Proceedings of the 2023 IEEE Belgrade PowerTech
Place of Publication	Piscataway
Publisher	IEEE
Pages	1-6
Number of pages	6
ISBN (Electronic)	978-1-6654-8778-8
ISBN (Print)	978-1-6654-8779-5
DOIs	https://doi.org/10.1109/PowerTech55446.2023.10202699
Publication status	Published - 2023
Event	2023 IEEE Belgrade PowerTech - Belgrade, Serbia Duration: 25 Jun 2023 → 29 Jun 2023

Publication series

Name	2023 IEEE Belgrade PowerTech, PowerTech 2023

Conference

Conference	2023 IEEE Belgrade PowerTech
Country/Territory	Serbia
City	Belgrade
Period	25/06/23 → 29/06/23

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Keywords

Q-learning
tile coding
energy management system
mixed-integer linear programming

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/PowerTech55446.2023.10202699

Optimal_Energy_Scheduling_of_Flexible_Industrial_Prosumers_via_Reinforcement_LearningFinal published version, 2.83 MB

Cite this

van den Bovenkamp, N., Giraldo, J. S., Salazar Duque, E. M., Vergara , P. P., Konstantinou, C., & Palensky, P. (2023). Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning. In Proceedings of the 2023 IEEE Belgrade PowerTech (pp. 1-6). (2023 IEEE Belgrade PowerTech, PowerTech 2023). IEEE. https://doi.org/10.1109/PowerTech55446.2023.10202699

@inproceedings{42fcf01b29f9462da8343f0158640409,

title = "Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning",

abstract = "This paper introduces an energy management system (EMS) aiming to minimize electricity operating costs using reinforcement learning (RL) with a linear function approximation. The proposed EMS uses a Q-learning with tile coding (QLTC) algorithm and is compared to a deterministic mixed-integer linear programming (MILP) with perfect forecast information. The comparison is performed using a case study on an industrial manufacturing company in the Netherlands, considering measured electricity consumption, PV generation, and wholesale electricity prices during one week of operation. The results show that the proposed EMS can adjust the prosumer's power consumption considering favorable prices. The electricity costs obtained using the QLTC algorithm are 99% close to those obtained with the MILP model. Furthermore, the results demonstrate that the QLTC model can generalize on previously learned control policies even in the case of missing data and can deploy actions 80% near to the MILP's optimal solution.",

keywords = "Q-learning, tile coding, energy management system, mixed-integer linear programming",

author = "{van den Bovenkamp}, Nick and Giraldo, {Juan S.} and {Salazar Duque}, {Edgar Mauricio} and Vergara, {Pedro P.} and Charalambos Konstantinou and Peter Palensky",

note = "Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.; 2023 IEEE Belgrade PowerTech ; Conference date: 25-06-2023 Through 29-06-2023",

year = "2023",

doi = "10.1109/PowerTech55446.2023.10202699",

language = "English",

isbn = "978-1-6654-8779-5",

series = "2023 IEEE Belgrade PowerTech, PowerTech 2023",

publisher = "IEEE",

pages = "1--6",

booktitle = "Proceedings of the 2023 IEEE Belgrade PowerTech",

address = "United States",

}

van den Bovenkamp, N, Giraldo, JS, Salazar Duque, EM, Vergara , PP, Konstantinou, C & Palensky, P 2023, Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning. in Proceedings of the 2023 IEEE Belgrade PowerTech. 2023 IEEE Belgrade PowerTech, PowerTech 2023, IEEE, Piscataway, pp. 1-6, 2023 IEEE Belgrade PowerTech, Belgrade, Serbia, 25/06/23. https://doi.org/10.1109/PowerTech55446.2023.10202699

Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning. / van den Bovenkamp, Nick ; Giraldo, Juan S.; Salazar Duque, Edgar Mauricio et al.
Proceedings of the 2023 IEEE Belgrade PowerTech. Piscataway: IEEE, 2023. p. 1-6 (2023 IEEE Belgrade PowerTech, PowerTech 2023).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning

AU - van den Bovenkamp, Nick

AU - Giraldo, Juan S.

AU - Salazar Duque, Edgar Mauricio

AU - Vergara , Pedro P.

AU - Konstantinou, Charalambos

AU - Palensky, Peter

N1 - Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

PY - 2023

Y1 - 2023

N2 - This paper introduces an energy management system (EMS) aiming to minimize electricity operating costs using reinforcement learning (RL) with a linear function approximation. The proposed EMS uses a Q-learning with tile coding (QLTC) algorithm and is compared to a deterministic mixed-integer linear programming (MILP) with perfect forecast information. The comparison is performed using a case study on an industrial manufacturing company in the Netherlands, considering measured electricity consumption, PV generation, and wholesale electricity prices during one week of operation. The results show that the proposed EMS can adjust the prosumer's power consumption considering favorable prices. The electricity costs obtained using the QLTC algorithm are 99% close to those obtained with the MILP model. Furthermore, the results demonstrate that the QLTC model can generalize on previously learned control policies even in the case of missing data and can deploy actions 80% near to the MILP's optimal solution.

AB - This paper introduces an energy management system (EMS) aiming to minimize electricity operating costs using reinforcement learning (RL) with a linear function approximation. The proposed EMS uses a Q-learning with tile coding (QLTC) algorithm and is compared to a deterministic mixed-integer linear programming (MILP) with perfect forecast information. The comparison is performed using a case study on an industrial manufacturing company in the Netherlands, considering measured electricity consumption, PV generation, and wholesale electricity prices during one week of operation. The results show that the proposed EMS can adjust the prosumer's power consumption considering favorable prices. The electricity costs obtained using the QLTC algorithm are 99% close to those obtained with the MILP model. Furthermore, the results demonstrate that the QLTC model can generalize on previously learned control policies even in the case of missing data and can deploy actions 80% near to the MILP's optimal solution.

KW - Q-learning

KW - tile coding

KW - energy management system

KW - mixed-integer linear programming

UR - http://www.scopus.com/inward/record.url?scp=85169463558&partnerID=8YFLogxK

U2 - 10.1109/PowerTech55446.2023.10202699

DO - 10.1109/PowerTech55446.2023.10202699

M3 - Conference contribution

SN - 978-1-6654-8779-5

T3 - 2023 IEEE Belgrade PowerTech, PowerTech 2023

SP - 1

EP - 6

BT - Proceedings of the 2023 IEEE Belgrade PowerTech

PB - IEEE

CY - Piscataway

T2 - 2023 IEEE Belgrade PowerTech

Y2 - 25 June 2023 through 29 June 2023

ER -

Optimal Energy Scheduling of Flexible Industrial Prosumers via Reinforcement Learning

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this