Markov Decision Processes in Practice

Richard J. Boucherie (Editor), Nico M. van Dijk (Editor)

Research output: Book/ReportBook editingAcademic

216 Downloads (Pure)


It is over 30 years ago since D.J. White started his series of surveys on practical applications of Markov decision processes (MDP), over 20 years after the phenomenal book by Martin Puterman on the theory of MDP, and over 10 years since Eugene A. Feinberg and Adam Shwartz published their Handbook of Markov Decision Processes: Methods and Applications. In the past decades, the practical development of MDP seemed to have come to a halt with the general perception that MDP is computationally prohibitive. Accordingly, MDP is deemed unrealistic and is out of scope for many operations research practitioners. In addition, MDP is hampered by its notational complications and its conceptual complexity. As a result, MDP is often only briefly covered in introductory operations research textbooks and courses. Recently developed approximation techniques supported by vastly increased numerical power have tackled part of the computational problems; see, e.g., Chaps. 2 and 3 of this handbook and the references therein. This handbook shows that a revival of MDP for practical purposes is justified for several reasons: 1. First and above all, the present-day numerical capabilities have enabled MDP to be invoked for real-life applications. 2. MDP allows to develop and formally support approximate and simple practical decision rules. 3. Last but not least, MDP’s probabilistic modeling of practical problems is a skill if not art by itself.
Original languageEnglish
Place of PublicationCham
ISBN (Electronic)978-3-319-47766-4
ISBN (Print)978-3-319-47764-0
Publication statusPublished - 2017

Publication series

NameInternational Series in Operations Research & Management Science
PublisherSpringer International Publishing
ISSN (Print)0884-8289
ISSN (Electronic)2214-7934


  • Markov decision process (MDP)


Dive into the research topics of 'Markov Decision Processes in Practice'. Together they form a unique fingerprint.

Cite this