Transient reward approximation for continuous-time Markov chains

Ernst Moritz Hahn, Holger Hermanns, Ralf Wimmer, Bernd Becker

Research output: Contribution to journalArticleAcademicpeer-review

3 Citations (Scopus)

Abstract

We are interested in the analysis of very large continuous-time Markov chains (CTMCs) with many distinct rates. Such models arise naturally in the context of reliability analysis, e.g., of computer network performability analysis, of power grids, of computer virus vulnerability, and in the study of crowd dynamics. We use abstraction techniques together with novel algorithms for the computation of bounds on the expected final and accumulated rewards in continuous-time Markov decision processes (CTMDPs). These ingredients are combined in a partly symbolic and partly explicit (symblicit) analysis approach. In particular, we circumvent the use of multi-terminal decision diagrams, because the latter do not work well if facing a large number of different rates. We demonstrate the practical applicability and efficiency of the approach on two case studies.

Original languageEnglish
Article number7163373
Pages (from-to)1254-1275
Number of pages22
JournalIEEE transactions on reliability
Volume64
Issue number4
DOIs
Publication statusPublished - 2015
Externally publishedYes

Keywords

  • Abstraction
  • Continuous-time Markov chains
  • Continuous-time Markov decision processes
  • Ordered binary decision diagrams
  • Symbolic methods

Fingerprint Dive into the research topics of 'Transient reward approximation for continuous-time Markov chains'. Together they form a unique fingerprint.

Cite this