Mungojerrie: Linear-Time Objectives in Model-Free Reinforcement Learning

Ernst Moritz Hahn; Mateo Perez; Sven Schewe; Fabio Somenzi; Ashutosh Trivedi; Dominik Wojtczak

doi:10.1007/978-3-031-30823-9_27

Mungojerrie: Linear-Time Objectives in Model-Free Reinforcement Learning

Ernst Moritz Hahn, Mateo Perez, Sven Schewe, Fabio Somenzi^*, Ashutosh Trivedi, Dominik Wojtczak

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

4 Citations (Scopus)

41 Downloads (Pure)

Abstract

Mungojerrie is an extensible tool that provides a framework to translate linear-time objectives into reward for reinforcement learning (RL). The tool provides convergent RL algorithms for stochastic games, reference implementations of existing reward translations for ω -regular objectives, and an internal probabilistic model checker for ω -regular objectives. This functionality is modular and operates on shared data structures, which enables fast development of new translation techniques. Mungojerrie supports finite models specified in PRISM and ω -automata specified in the HOA format, with an integrated command line interface to external linear temporal logic translators. Mungojerrie is distributed with a set of benchmarks for ω -regular objectives in RL.

Original language	English
Title of host publication	Tools and Algorithms for the Construction and Analysis of Systems - 29th International Conference, TACAS 2023, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022, Proceedings
Editors	Sriram Sankaranarayanan, Natasha Sharygina
Publisher	Springer
Pages	527-545
Number of pages	19
ISBN (Print)	9783031308222
DOIs	https://doi.org/10.1007/978-3-031-30823-9_27
Publication status	Published - 22 Apr 2023
Event	29th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2023 - Paris, France Duration: 22 Apr 2023 → 27 Apr 2023 Conference number: 29

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13993 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	29th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2023
Abbreviated title	TACAS 2023
Country/Territory	France
City	Paris
Period	22/04/23 → 27/04/23

Access to Document

10.1007/978-3-031-30823-9_27Licence: CC BY

978-3-031-30823-9_27Final published version, 764 KBLicence: CC BY

Cite this

Hahn, E. M., Perez, M., Schewe, S., Somenzi, F., Trivedi, A., & Wojtczak, D. (2023). Mungojerrie: Linear-Time Objectives in Model-Free Reinforcement Learning. In S. Sankaranarayanan, & N. Sharygina (Eds.), Tools and Algorithms for the Construction and Analysis of Systems - 29th International Conference, TACAS 2023, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022, Proceedings (pp. 527-545). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13993 LNCS). Springer. https://doi.org/10.1007/978-3-031-30823-9_27

Hahn, Ernst Moritz ; Perez, Mateo ; Schewe, Sven et al. / Mungojerrie : Linear-Time Objectives in Model-Free Reinforcement Learning. Tools and Algorithms for the Construction and Analysis of Systems - 29th International Conference, TACAS 2023, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022, Proceedings. editor / Sriram Sankaranarayanan ; Natasha Sharygina. Springer, 2023. pp. 527-545 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{c9aec96c9de14009bf5873d6162f24f4,

title = "Mungojerrie: Linear-Time Objectives in Model-Free Reinforcement Learning",

abstract = "Mungojerrie is an extensible tool that provides a framework to translate linear-time objectives into reward for reinforcement learning (RL). The tool provides convergent RL algorithms for stochastic games, reference implementations of existing reward translations for ω -regular objectives, and an internal probabilistic model checker for ω -regular objectives. This functionality is modular and operates on shared data structures, which enables fast development of new translation techniques. Mungojerrie supports finite models specified in PRISM and ω -automata specified in the HOA format, with an integrated command line interface to external linear temporal logic translators. Mungojerrie is distributed with a set of benchmarks for ω -regular objectives in RL.",

author = "Hahn, {Ernst Moritz} and Mateo Perez and Sven Schewe and Fabio Somenzi and Ashutosh Trivedi and Dominik Wojtczak",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s).; 29th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2023, TACAS 2023 ; Conference date: 22-04-2023 Through 27-04-2023",

year = "2023",

month = apr,

day = "22",

doi = "10.1007/978-3-031-30823-9_27",

language = "English",

isbn = "9783031308222",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "527--545",

editor = "Sriram Sankaranarayanan and Natasha Sharygina",

booktitle = "Tools and Algorithms for the Construction and Analysis of Systems - 29th International Conference, TACAS 2023, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022, Proceedings",

address = "Germany",

}

Hahn, EM, Perez, M, Schewe, S, Somenzi, F, Trivedi, A & Wojtczak, D 2023, Mungojerrie: Linear-Time Objectives in Model-Free Reinforcement Learning. in S Sankaranarayanan & N Sharygina (eds), Tools and Algorithms for the Construction and Analysis of Systems - 29th International Conference, TACAS 2023, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13993 LNCS, Springer, pp. 527-545, 29th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2023, Paris, France, 22/04/23. https://doi.org/10.1007/978-3-031-30823-9_27

Mungojerrie: Linear-Time Objectives in Model-Free Reinforcement Learning. / Hahn, Ernst Moritz; Perez, Mateo; Schewe, Sven et al.
Tools and Algorithms for the Construction and Analysis of Systems - 29th International Conference, TACAS 2023, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022, Proceedings. ed. / Sriram Sankaranarayanan; Natasha Sharygina. Springer, 2023. p. 527-545 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13993 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Mungojerrie

T2 - 29th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2023

AU - Hahn, Ernst Moritz

AU - Perez, Mateo

AU - Schewe, Sven

AU - Somenzi, Fabio

AU - Trivedi, Ashutosh

AU - Wojtczak, Dominik

N1 - Conference code: 29

PY - 2023/4/22

Y1 - 2023/4/22

N2 - Mungojerrie is an extensible tool that provides a framework to translate linear-time objectives into reward for reinforcement learning (RL). The tool provides convergent RL algorithms for stochastic games, reference implementations of existing reward translations for ω -regular objectives, and an internal probabilistic model checker for ω -regular objectives. This functionality is modular and operates on shared data structures, which enables fast development of new translation techniques. Mungojerrie supports finite models specified in PRISM and ω -automata specified in the HOA format, with an integrated command line interface to external linear temporal logic translators. Mungojerrie is distributed with a set of benchmarks for ω -regular objectives in RL.

AB - Mungojerrie is an extensible tool that provides a framework to translate linear-time objectives into reward for reinforcement learning (RL). The tool provides convergent RL algorithms for stochastic games, reference implementations of existing reward translations for ω -regular objectives, and an internal probabilistic model checker for ω -regular objectives. This functionality is modular and operates on shared data structures, which enables fast development of new translation techniques. Mungojerrie supports finite models specified in PRISM and ω -automata specified in the HOA format, with an integrated command line interface to external linear temporal logic translators. Mungojerrie is distributed with a set of benchmarks for ω -regular objectives in RL.

UR - http://www.scopus.com/inward/record.url?scp=85161403203&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-30823-9_27

DO - 10.1007/978-3-031-30823-9_27

M3 - Conference contribution

AN - SCOPUS:85161403203

SN - 9783031308222

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 527

EP - 545

BT - Tools and Algorithms for the Construction and Analysis of Systems - 29th International Conference, TACAS 2023, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022, Proceedings

A2 - Sankaranarayanan, Sriram

A2 - Sharygina, Natasha

PB - Springer

Y2 - 22 April 2023 through 27 April 2023

ER -

Hahn EM, Perez M, Schewe S, Somenzi F, Trivedi A, Wojtczak D. Mungojerrie: Linear-Time Objectives in Model-Free Reinforcement Learning. In Sankaranarayanan S, Sharygina N, editors, Tools and Algorithms for the Construction and Analysis of Systems - 29th International Conference, TACAS 2023, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2022, Proceedings. Springer. 2023. p. 527-545. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-30823-9_27

Mungojerrie: Linear-Time Objectives in Model-Free Reinforcement Learning

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this