Attention-Based Temporal Reinforcement Learning for Energy System Control

Reinforcement Learning (RL) agents provide intelligent control for energy systems through direct interaction with their environment. While RL agents can learn system dynamics from historical data, they often struggle to capture temporal patterns, particularly in high-fluctuating conditions. This paper introduces temporal RL agents, which can utilize forecasted patterns in the control of energy systems. The proposed agent employs an attention-based temporal embedding module to extract relevant information from forecasted time series. This information is represented as temporal embeddings, which enable the agent to consider future system patterns when making control decisions. The generalization and adaptability of the temporal RL agent are evaluated using fluctuating patterns from the Netherlands, including time series of generation, load, price, and CO₂ emission. Moreover, a SHAP-based feature analysis highlights the importance of temporal features on agent's decisions.