End-to-End Learning with Multiple Modalities for System-Optimised Renewables Nowcasting

Research output: Chapter in Book/Conference proceedings/Edited volumeConference contributionScientificpeer-review

1 Citation (Scopus)
5 Downloads (Pure)


With the increasing penetration of renewable power sources such as wind and solar, accurate short-term, nowcasting renewable power prediction is becoming increasingly important. This paper investigates the multi-modal (MM) learning and end-to-end (E2E) learning for nowcasting renewable power as an intermediate to energy management systems. MM combines features from all-sky imagery and meteorological sensor data as two modalities to predict renewable power generation that otherwise could not be combined effectively. The combined, predicted values are then input to a differentiable optimal power flow (OPF) formulation simulating the energy management. For the first time, MM is combined with E2E training of the model that minimises the expected total system cost. The case study tests the proposed methodology on the real sky and meteorological data from the Netherlands. In our study, the proposed MM- E2E model reduced system cost by 30% compared to uni-modal baselines.
Original languageEnglish
Title of host publicationProceedings of the 2023 IEEE Belgrade PowerTech
Place of PublicationPiscataway
Number of pages8
ISBN (Electronic)978-1-6654-8778-8
ISBN (Print)978-1-6654-8779-5
Publication statusPublished - 2023
Event2023 IEEE Belgrade PowerTech - Belgrade, Serbia
Duration: 25 Jun 202329 Jun 2023

Publication series

Name2023 IEEE Belgrade PowerTech, PowerTech 2023


Conference2023 IEEE Belgrade PowerTech

Bibliographical note

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care
Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.


  • multi-modal learning
  • end-to-end learning
  • optimal power flow
  • power forecasting


Dive into the research topics of 'End-to-End Learning with Multiple Modalities for System-Optimised Renewables Nowcasting'. Together they form a unique fingerprint.

Cite this