TRANSFORMER MODELI MAŠINSKOG UČENJA ZA PREDVIĐANJE VREMENSKIH SERIJA U ELEKTROENERGETICI

37. savetovanje CIGRE Srbija (2025) SIGURNOST, STABILNOST, POUZDANOST I RESILIENCE ELEKTROENERGETSKOG SISTEMA MULTISEKTORSKO POVEZIVANJE U ENERGETICI I PRIVREDI – D2-02

AUTOR(I) / AUTHOR(S): Saša Milić, Miša Kožicić, Luka Ivanović, Miroslav Dragićević

 

Download Full Pdf   

DOI:  10.46793/CIGRE37.D2.02

SAŽETAK / ABSTRACT:

Time series forecasting in the electricity power sector represents a key challenge for production prediction, management optimization, maintenance strategy selection for capital equipment, and planning of electricity generation and consumption. Transformer models have become the dominant architecture in the field of natural language processing, but their successful application in time series analysis has opened new opportunities for improving prediction accuracy. This paper consists of two interconnected parts. The first part provides a detailed overview of two fundamental transformer model architectures. The second part analyses their characteristics through practical tasks related to long-term time series forecasting. A detailed examination is conducted on the impact of various model hyper parameters, the size of input data window, the prediction horizon, and attention mechanisms.

The proposed models are validated using a dataset on energy production and consumption from solar renewable sources. Model performance is evaluated using several types of standard metrics. The experimental results with transformer models justify their application in long-term time-series predictions. The analysis of the obtained results enables a discussion on the potential applications of these models in the power sector, including production forecasting from renewable sources and the efficient integration of these renewable energy sources into the power system. The conclusion strongly emphasizes the significance of future applications of transformer-based machine learning models in almost all power system branches.

KLJUČNE REČI / KEYWORDS:

Machine learning, transformer models, time series, renewable energy sources.

PROJEKAT / ACKNOWLEDGEMENT:

Ovaj  rad je  podržalo Ministarstvo nauke, tehnološkog razvoja i inovacija Republike Srbije kroz Ugovor o realizaciji i finansiranju naučnoistraživačkog rada NIO u 2025. godini.

LITERATURA / REFERENCES:

  • Ivanović Luka, Milić Saša D, Sokolović Živko, Rakić Aleksandar, “Komparativna analiza dubokih neuronskih mreža i algoritama sa pojačanjem gradijenta u dugoročnom predviđanju snage vetra”, Zbornik radova, Elektrotehnički institut “Nikola Tesla”, 2024, ISSN 0350−8528. https://doi.org/10.5937/zeint34-51258
  • Saša Milić, Luka Ivanović, Žarko Janda, Miroslav Dragićević, ”Transformer Machine Learning Models for Time Series Electricity Power Forecasting”, 24rd International Symposium INFOTEH-JAHORINA, 19-21 March 2025.
  • https://infoteh.etf.ues.rs.ba/radovi.php
  • B. Gao, X. Huang, J. Shi, Y. Tai, J. Zhang, “Hourly forecasting of solar irradiance based on CEEMDAN and multi-strategy CNN-LSTM neural networks,” Renewable Energy, vol. 162, 2020, pp. 1665-1683.
  • F. Yang, Z. Chen, Y. Liang, “Precise solar radiation forecasting for sustainable energy integration – A hybrid model for day-ahead power and hydrogen production,” Renewable Energy, vol. 237, part C, 2024, 121732.
  • Skup podataka: Renewable Power Gen Prediction with Time Series, Renewable.csv. Licenca: Apache 2.0. Dostupno na sajtu: https://www.kaggle.com/datasets/sheemazain/renewable-power-generation-weather-condition-2024
  • H. Iftikhar, S. M. Gonzales; J. Zywiołek, J. L. López-Gonzales, “Electricity Demand Forecasting Using a Novel Time Series Ensemble Technique,” IEEE Access, vol. 12, 2024, pp. 88963 – 88975.
  • L. Zhang, D. Jánošík, “Enhanced short-term load forecasting with hybrid machine learning models: CatBoost and XGBoost approaches,“ Expert Systems with Applications, vol. 241, 2024, 122686. https://doi.org/10.1016/j.eswa.2023.122686
  • A. Vaswani, et al., “Attention Is All You Need,” In the Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS), 2017, Long Beach, CA, USA. https://doi.org/10.48550/arXiv.1706.03762
  • B. Lim, S. Ö. Arık, N. Loeff, T. Pfister, “Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting,” International Journal of Forecasting, vol. 37, 2021, pp. 1748-1764.