Enabling cooperative behavior for building demand response based on extended joint action learning

Luis A. Hurtado Muñoz, Elena Mocanu, Phuong H. Nguyen, Madeleine Gibescu, René I.G. Kamphuis

Research output: Contribution to journalArticleAcademicpeer-review

9 Citations (Scopus)

Abstract

This paper explores the use of distributed intelligence to assist the integration of the demand as a flexible resource, to mitigate the emerging uncertainty in the power system, while fulfilling the customer’s local needs, i.e., comfort
management. More exactly, our contribution is two-fold. Firstly, we propose a novel cooperative and decentralized reinforcement learning method, dubbed extended joint action learning (eJAL). Secondly, we perform a comparison between eJAL to noncooperative decentralized decision making strategies, i.e., Qlearning, and a centralized game theoretic approach, i.e., Nash n-player game. This comparison has been conducted on the basis of grid support effectiveness and the loss of comfort for each customer. Various metrics were used to analyze the advantages and disadvantages of each method. We demonstrated that a range of flexibility requests can be met by providing an optimal energy portfolio of buildings without substantially violating comfort constraints. Moreover, we showed that the proposed eJAL method achieves the highest fairness index.
Original languageEnglish
Article number8039194
Pages (from-to)127-136
Number of pages10
JournalIEEE transactions on industrial informatics
Volume14
Issue number1
DOIs
Publication statusPublished - 1 Jan 2018
Externally publishedYes

Fingerprint

Reinforcement learning
Decision making
Uncertainty

Keywords

  • Reinforcement Learning
  • collaborative learning

Cite this

Hurtado Muñoz, Luis A. ; Mocanu, Elena ; Nguyen, Phuong H. ; Gibescu, Madeleine ; Kamphuis, René I.G. / Enabling cooperative behavior for building demand response based on extended joint action learning. In: IEEE transactions on industrial informatics. 2018 ; Vol. 14, No. 1. pp. 127-136.
@article{b6ba98d085704c4b831fb95681fb1362,
title = "Enabling cooperative behavior for building demand response based on extended joint action learning",
abstract = "This paper explores the use of distributed intelligence to assist the integration of the demand as a flexible resource, to mitigate the emerging uncertainty in the power system, while fulfilling the customer’s local needs, i.e., comfortmanagement. More exactly, our contribution is two-fold. Firstly, we propose a novel cooperative and decentralized reinforcement learning method, dubbed extended joint action learning (eJAL). Secondly, we perform a comparison between eJAL to noncooperative decentralized decision making strategies, i.e., Qlearning, and a centralized game theoretic approach, i.e., Nash n-player game. This comparison has been conducted on the basis of grid support effectiveness and the loss of comfort for each customer. Various metrics were used to analyze the advantages and disadvantages of each method. We demonstrated that a range of flexibility requests can be met by providing an optimal energy portfolio of buildings without substantially violating comfort constraints. Moreover, we showed that the proposed eJAL method achieves the highest fairness index.",
keywords = "Reinforcement Learning, collaborative learning",
author = "{Hurtado Mu{\~n}oz}, {Luis A.} and Elena Mocanu and Nguyen, {Phuong H.} and Madeleine Gibescu and Kamphuis, {Ren{\'e} I.G.}",
year = "2018",
month = "1",
day = "1",
doi = "10.1109/TII.2017.2753408",
language = "English",
volume = "14",
pages = "127--136",
journal = "IEEE transactions on industrial informatics",
issn = "1551-3203",
publisher = "IEEE Computer Society",
number = "1",

}

Enabling cooperative behavior for building demand response based on extended joint action learning. / Hurtado Muñoz, Luis A. ; Mocanu, Elena; Nguyen, Phuong H.; Gibescu, Madeleine; Kamphuis, René I.G.

In: IEEE transactions on industrial informatics, Vol. 14, No. 1, 8039194, 01.01.2018, p. 127-136.

Research output: Contribution to journalArticleAcademicpeer-review

TY - JOUR

T1 - Enabling cooperative behavior for building demand response based on extended joint action learning

AU - Hurtado Muñoz, Luis A.

AU - Mocanu, Elena

AU - Nguyen, Phuong H.

AU - Gibescu, Madeleine

AU - Kamphuis, René I.G.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - This paper explores the use of distributed intelligence to assist the integration of the demand as a flexible resource, to mitigate the emerging uncertainty in the power system, while fulfilling the customer’s local needs, i.e., comfortmanagement. More exactly, our contribution is two-fold. Firstly, we propose a novel cooperative and decentralized reinforcement learning method, dubbed extended joint action learning (eJAL). Secondly, we perform a comparison between eJAL to noncooperative decentralized decision making strategies, i.e., Qlearning, and a centralized game theoretic approach, i.e., Nash n-player game. This comparison has been conducted on the basis of grid support effectiveness and the loss of comfort for each customer. Various metrics were used to analyze the advantages and disadvantages of each method. We demonstrated that a range of flexibility requests can be met by providing an optimal energy portfolio of buildings without substantially violating comfort constraints. Moreover, we showed that the proposed eJAL method achieves the highest fairness index.

AB - This paper explores the use of distributed intelligence to assist the integration of the demand as a flexible resource, to mitigate the emerging uncertainty in the power system, while fulfilling the customer’s local needs, i.e., comfortmanagement. More exactly, our contribution is two-fold. Firstly, we propose a novel cooperative and decentralized reinforcement learning method, dubbed extended joint action learning (eJAL). Secondly, we perform a comparison between eJAL to noncooperative decentralized decision making strategies, i.e., Qlearning, and a centralized game theoretic approach, i.e., Nash n-player game. This comparison has been conducted on the basis of grid support effectiveness and the loss of comfort for each customer. Various metrics were used to analyze the advantages and disadvantages of each method. We demonstrated that a range of flexibility requests can be met by providing an optimal energy portfolio of buildings without substantially violating comfort constraints. Moreover, we showed that the proposed eJAL method achieves the highest fairness index.

KW - Reinforcement Learning

KW - collaborative learning

U2 - 10.1109/TII.2017.2753408

DO - 10.1109/TII.2017.2753408

M3 - Article

VL - 14

SP - 127

EP - 136

JO - IEEE transactions on industrial informatics

JF - IEEE transactions on industrial informatics

SN - 1551-3203

IS - 1

M1 - 8039194

ER -