Multi-cost Bounded Tradeoff Analysis in MDP

Arnd Hartmanns*, Sebastian Junges, Joost-Pieter Katoen, Tim Quatmann

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

14 Citations (Scopus)
84 Downloads (Pure)

Abstract

We provide a memory-efficient algorithm for multi-objective model checking problems on Markov decision processes (MDPs) with multiple cost structures. The key problem at hand is to check whether there exists a scheduler for a given MDP such that all objectives over cost vectors are fulfilled. We cover multi-objective reachability and expected cost objectives, and combinations thereof. We further transfer approaches for computing quantiles over single cost bounds to the multi-cost case and highlight the ensuing challenges. An empirical evaluation shows the scalability of our new approach both in terms of memory consumption and runtime. We discuss the need for more detailed visual presentations of results beyond Pareto curves and present a first visualisation approach that exploits all the available information from the algorithm to support decision makers.
Original languageEnglish
Pages (from-to)1483-1522
Number of pages40
JournalJournal of automated reasoning
Volume64
Issue number7
Early online date28 Jul 2020
DOIs
Publication statusPublished - Oct 2020

Keywords

  • Markov decision process
  • Multi-objective verification
  • Pareto-optimal strategies
  • Cost-bounded reachability
  • Expected rewards
  • Probabilistic model checking

Fingerprint

Dive into the research topics of 'Multi-cost Bounded Tradeoff Analysis in MDP'. Together they form a unique fingerprint.

Cite this