Embedding Temporal Awareness in Reinforcement Learning Models for Energy System Control

Amirhossein Heydarian Ardakani; Johann Hurink; Ian Gibson; Elham Shirazi

doi:10.1145/3679240.3734686

Embedding Temporal Awareness in Reinforcement Learning Models for Energy System Control

Amirhossein Heydarian Ardakani^*
, Johann Hurink
, Ian Gibson
, Elham Shirazi

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

1 Citation (Scopus)

9 Downloads (Pure)

Abstract

A reinforcement learning (RL) agent can capture energy system dynamics from historical data to improve its decision-making. However, the RL agent struggles to fully understand temporal patterns in scenarios with high fluctuations. This study addresses this gap by embedding temporal awareness into RL agents for energy system control. The proposed approach integrates generation and load time series into the state representation using a temporal embedding module based on long short-term memory (LSTM). The method is evaluated on highly variable patterns from the Netherlands to demonstrate the performance of temporal RL decision-making in different conditions.

Original language	English
Title of host publication	E-ENERGY 2025 - Proceedings of the 2025 16th ACM International Conference on Future and Sustainable Energy Systems
Publisher	Association for Computing Machinery
Pages	1002-1004
Number of pages	3
ISBN (Electronic)	9798400711251
DOIs	https://doi.org/10.1145/3679240.3734686
Publication status	Published - 16 Jun 2025
Event	16th ACM International Conference on Future and Sustainable Energy Systems, ACM E-Energy 2025 - Nhow Rotterdam Hotel, Rotterdam, Netherlands Duration: 17 Jun 2025 → 20 Jun 2025 Conference number: 16 https://energy.acm.org/conferences/eenergy/2025/

Conference

Conference	16th ACM International Conference on Future and Sustainable Energy Systems, ACM E-Energy 2025
Abbreviated title	ACM E-Energy 2025
Country/Territory	Netherlands
City	Rotterdam
Period	17/06/25 → 20/06/25
Internet address	https://energy.acm.org/conferences/eenergy/2025/

Keywords

Energy System Control
Long Short-Term Memory
Reinforcement Learning
Sequential Decision-making
Temporal Awareness

Access to Document

10.1145/3679240.3734686Licence: CC BY

3679240.3734686Final published version, 846 KBLicence: CC BY

Cite this

@inproceedings{a0cdbeb4f7554a5798bf6e0814a2692a,

title = "Embedding Temporal Awareness in Reinforcement Learning Models for Energy System Control",

abstract = "A reinforcement learning (RL) agent can capture energy system dynamics from historical data to improve its decision-making. However, the RL agent struggles to fully understand temporal patterns in scenarios with high fluctuations. This study addresses this gap by embedding temporal awareness into RL agents for energy system control. The proposed approach integrates generation and load time series into the state representation using a temporal embedding module based on long short-term memory (LSTM). The method is evaluated on highly variable patterns from the Netherlands to demonstrate the performance of temporal RL decision-making in different conditions.",

keywords = "Energy System Control, Long Short-Term Memory, Reinforcement Learning, Sequential Decision-making, Temporal Awareness",

author = "Ardakani, \{Amirhossein Heydarian\} and Johann Hurink and Ian Gibson and Elham Shirazi",

note = "Publisher Copyright: {\textcopyright} 2025 Copyright held by the owner/author(s).; 16th ACM International Conference on Future and Sustainable Energy Systems, ACM E-Energy 2025, ACM E-Energy 2025 ; Conference date: 17-06-2025 Through 20-06-2025",

year = "2025",

month = jun,

day = "16",

doi = "10.1145/3679240.3734686",

language = "English",

pages = "1002--1004",

booktitle = "E-ENERGY 2025 - Proceedings of the 2025 16th ACM International Conference on Future and Sustainable Energy Systems",

publisher = "Association for Computing Machinery",

address = "United States",

url = "https://energy.acm.org/conferences/eenergy/2025/",

}

Ardakani, AH, Hurink, J, Gibson, I & Shirazi, E 2025, Embedding Temporal Awareness in Reinforcement Learning Models for Energy System Control. in E-ENERGY 2025 - Proceedings of the 2025 16th ACM International Conference on Future and Sustainable Energy Systems. Association for Computing Machinery, pp. 1002-1004, 16th ACM International Conference on Future and Sustainable Energy Systems, ACM E-Energy 2025, Rotterdam, Netherlands, 17/06/25. https://doi.org/10.1145/3679240.3734686

Embedding Temporal Awareness in Reinforcement Learning Models for Energy System Control. / Ardakani, Amirhossein Heydarian; Hurink, Johann; Gibson, Ian et al.
E-ENERGY 2025 - Proceedings of the 2025 16th ACM International Conference on Future and Sustainable Energy Systems. Association for Computing Machinery, 2025. p. 1002-1004.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Embedding Temporal Awareness in Reinforcement Learning Models for Energy System Control

AU - Ardakani, Amirhossein Heydarian

AU - Hurink, Johann

AU - Gibson, Ian

AU - Shirazi, Elham

N1 - Conference code: 16

PY - 2025/6/16

Y1 - 2025/6/16

N2 - A reinforcement learning (RL) agent can capture energy system dynamics from historical data to improve its decision-making. However, the RL agent struggles to fully understand temporal patterns in scenarios with high fluctuations. This study addresses this gap by embedding temporal awareness into RL agents for energy system control. The proposed approach integrates generation and load time series into the state representation using a temporal embedding module based on long short-term memory (LSTM). The method is evaluated on highly variable patterns from the Netherlands to demonstrate the performance of temporal RL decision-making in different conditions.

AB - A reinforcement learning (RL) agent can capture energy system dynamics from historical data to improve its decision-making. However, the RL agent struggles to fully understand temporal patterns in scenarios with high fluctuations. This study addresses this gap by embedding temporal awareness into RL agents for energy system control. The proposed approach integrates generation and load time series into the state representation using a temporal embedding module based on long short-term memory (LSTM). The method is evaluated on highly variable patterns from the Netherlands to demonstrate the performance of temporal RL decision-making in different conditions.

KW - Energy System Control

KW - Long Short-Term Memory

KW - Reinforcement Learning

KW - Sequential Decision-making

KW - Temporal Awareness

UR - https://www.scopus.com/pages/publications/105016359389

U2 - 10.1145/3679240.3734686

DO - 10.1145/3679240.3734686

M3 - Conference contribution

AN - SCOPUS:105016359389

SP - 1002

EP - 1004

BT - E-ENERGY 2025 - Proceedings of the 2025 16th ACM International Conference on Future and Sustainable Energy Systems

PB - Association for Computing Machinery

T2 - 16th ACM International Conference on Future and Sustainable Energy Systems, ACM E-Energy 2025

Y2 - 17 June 2025 through 20 June 2025

ER -

Embedding Temporal Awareness in Reinforcement Learning Models for Energy System Control

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this