Reachability analysis for incomplete networks of Markov decision processes

Ralf Wimmer; Ernst Moritz Hahn; Holger Hermanns; Bernd Becker

doi:10.1109/MEMCOD.2011.5970522

Reachability analysis for incomplete networks of Markov decision processes

Ralf Wimmer^*, Ernst Moritz Hahn, Holger Hermanns, Bernd Becker

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Assume we have a network of discrete-time Markov decision processes (MDPs) which synchronize via common actions. We investigate how to compute probability measures in case the structure of some of the component MDPs (so-called blackbox MDPs) is not known. We then extend this computation to work on networks of MDPs that share integer data variables of finite domain. We use a protocol which spreads information within a network as a case study to show the feasibility and effectiveness of our approach.

Original language	English
Title of host publication	9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011
Pages	151-160
Number of pages	10
DOIs	https://doi.org/10.1109/MEMCOD.2011.5970522
Publication status	Published - 2011
Externally published	Yes
Event	9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011 - Cambridge, United Kingdom Duration: 11 Jul 2011 → 13 Jul 2011 Conference number: 9

Conference

Conference	9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011
Abbreviated title	MEMOCODE 2011
Country/Territory	United Kingdom
City	Cambridge
Period	11/07/11 → 13/07/11

Access to Document

10.1109/MEMCOD.2011.5970522

Cite this

@inproceedings{20c065aefad3495fac9e169ac0345242,

title = "Reachability analysis for incomplete networks of Markov decision processes",

abstract = "Assume we have a network of discrete-time Markov decision processes (MDPs) which synchronize via common actions. We investigate how to compute probability measures in case the structure of some of the component MDPs (so-called blackbox MDPs) is not known. We then extend this computation to work on networks of MDPs that share integer data variables of finite domain. We use a protocol which spreads information within a network as a case study to show the feasibility and effectiveness of our approach.",

author = "Ralf Wimmer and Hahn, {Ernst Moritz} and Holger Hermanns and Bernd Becker",

year = "2011",

doi = "10.1109/MEMCOD.2011.5970522",

language = "English",

isbn = "9781457701160",

pages = "151--160",

booktitle = "9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011",

note = "9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011, MEMOCODE 2011 ; Conference date: 11-07-2011 Through 13-07-2011",

}

Wimmer, R, Hahn, EM, Hermanns, H & Becker, B 2011, Reachability analysis for incomplete networks of Markov decision processes. in 9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011., 5970522, pp. 151-160, 9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011, Cambridge, United Kingdom, 11/07/11. https://doi.org/10.1109/MEMCOD.2011.5970522

Reachability analysis for incomplete networks of Markov decision processes. / Wimmer, Ralf; Hahn, Ernst Moritz; Hermanns, Holger et al.
9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011. 2011. p. 151-160 5970522.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Reachability analysis for incomplete networks of Markov decision processes

AU - Wimmer, Ralf

AU - Hahn, Ernst Moritz

AU - Hermanns, Holger

AU - Becker, Bernd

N1 - Conference code: 9

PY - 2011

Y1 - 2011

N2 - Assume we have a network of discrete-time Markov decision processes (MDPs) which synchronize via common actions. We investigate how to compute probability measures in case the structure of some of the component MDPs (so-called blackbox MDPs) is not known. We then extend this computation to work on networks of MDPs that share integer data variables of finite domain. We use a protocol which spreads information within a network as a case study to show the feasibility and effectiveness of our approach.

AB - Assume we have a network of discrete-time Markov decision processes (MDPs) which synchronize via common actions. We investigate how to compute probability measures in case the structure of some of the component MDPs (so-called blackbox MDPs) is not known. We then extend this computation to work on networks of MDPs that share integer data variables of finite domain. We use a protocol which spreads information within a network as a case study to show the feasibility and effectiveness of our approach.

UR - http://www.scopus.com/inward/record.url?scp=80052110660&partnerID=8YFLogxK

U2 - 10.1109/MEMCOD.2011.5970522

DO - 10.1109/MEMCOD.2011.5970522

M3 - Conference contribution

AN - SCOPUS:80052110660

SN - 9781457701160

SP - 151

EP - 160

BT - 9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011

T2 - 9th ACM/IEEE International Conference on Formal Methods and Models for Codesign, MEMOCODE 2011

Y2 - 11 July 2011 through 13 July 2011

ER -

Reachability analysis for incomplete networks of Markov decision processes

Abstract

Conference

Access to Document

Other files and links

Fingerprint

Cite this