Learning optimal decisions for stochastic hybrid systems

Mathis Niehage, Arnd Hartmanns, Anne Remke

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

14 Downloads (Pure)

Abstract

We apply reinforcement learning to approximate the optimal probability that a stochastic hybrid system satisfies a temporal logic formula. We consider systems with (non)linear continuous dynamics, random events following general continuous probability distributions, and discrete nondeterministic choices. We present a discretized view of states to the learner, but simulate the continuous system. Once we have learned a near-optimal scheduler resolving the choices, we use statistical model checking to estimate its probability of satisfying the formula. We implemented the approach using Q-learning in the tools HYPEG and modes, which support Petri net- and hybrid automata-based models, respectively. Via two case studies, we show the feasibility of the approach, and compare its performance and effectiveness to existing analytical techniques for a linear model. We find that our new approach quickly finds near-optimal prophetic as well as non-prophetic schedulers, which maximize or minimize the probability that a specific signal temporal logic property is satisfied.
Original languageEnglish
Title of host publicationMEMOCODE '21
Subtitle of host publicationPrtoceedings of the 19th ACM-IEEE International Conference on Formal Methods and Models for System Design, Virtual Event, China, November 20-22, 2021
EditorsS. Arun-Kumar, Dominique Méry, Indranil Saha, Lijun Zhang
PublisherACM Press
Pages44-55
Number of pages12
ISBN (Print)978-1-4503-9127-6
DOIs
Publication statusPublished - 20 Nov 2021
Event19th ACM-IEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2021 - Virtual Event
Duration: 20 Nov 202122 Nov 2021
Conference number: 19

Conference

Conference19th ACM-IEEE International Conference on Formal Methods and Models for System Design, MEMOCODE 2021
Abbreviated titleMEMOCODE 2021
CityVirtual Event
Period20/11/2122/11/21

Fingerprint

Dive into the research topics of 'Learning optimal decisions for stochastic hybrid systems'. Together they form a unique fingerprint.

Cite this