Model-Based Testing of Probabilistic Systems

Marcus Gerhold; Mariëlle Ida Antoinette Stoelinga

doi:10.1007/978-3-662-49665-7_15

Model-Based Testing of Probabilistic Systems

Marcus Gerhold, Mariëlle Ida Antoinette Stoelinga

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

8 Citations (Scopus)

95 Downloads (Pure)

Abstract

This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

Original language	English
Title of host publication	Fundamental Approaches to Software Engineering
Subtitle of host publication	19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings
Editors	Perdita Stevens, Andzej Wasowski
Place of Publication	Heidelberg-Berlin
Publisher	Springer
Pages	251-268
Number of pages	18
ISBN (Electronic)	978-3-662-49665-7
ISBN (Print)	978-3-662-49664-0
DOIs	https://doi.org/10.1007/978-3-662-49665-7_15
Publication status	Published - Apr 2016
Event	19th International Conference on Fundamental Approaches to Software Engineering, FASE 2016 - Eindhoven University of Technology, Eindhoven, Netherlands Duration: 2 Apr 2016 → 8 Apr 2016 Conference number: 19 https://www.etaps.org/index.php/2016/fase

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer Verlag
Volume	9633
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	19th International Conference on Fundamental Approaches to Software Engineering, FASE 2016
Abbreviated title	FASE
Country	Netherlands
City	Eindhoven
Period	2/04/16 → 8/04/16
Internet address	https://www.etaps.org/index.php/2016/fase

Keywords

FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS
EC Grant Agreement nr.: FP7/318490
Statistical hypothesis testing
Trace distribution
False rejection
Probabilistic automaton
Finite path

Access to Document

10.1007/978-3-662-49665-7_15

Fingerprint Dive into the research topics of 'Model-Based Testing of Probabilistic Systems'. Together they form a unique fingerprint.

Cite this

Gerhold, M., & Stoelinga, M. I. A. (2016). Model-Based Testing of Probabilistic Systems. In P. Stevens, & A. Wasowski (Eds.), Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings (pp. 251-268). (Lecture Notes in Computer Science; Vol. 9633). Heidelberg-Berlin: Springer. https://doi.org/10.1007/978-3-662-49665-7_15

Gerhold, Marcus ; Stoelinga, Mariëlle Ida Antoinette. / Model-Based Testing of Probabilistic Systems. Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. editor / Perdita Stevens ; Andzej Wasowski. Heidelberg-Berlin : Springer, 2016. pp. 251-268 (Lecture Notes in Computer Science).

@inproceedings{672e674a112047888e8102eddd39f562,

title = "Model-Based Testing of Probabilistic Systems",

abstract = "This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.",

keywords = "FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS, EC Grant Agreement nr.: FP7/318490, Statistical hypothesis testing, Trace distribution, False rejection, Probabilistic automaton, Finite path",

author = "Marcus Gerhold and Stoelinga, {Mari{\"e}lle Ida Antoinette}",

year = "2016",

month = "4",

doi = "10.1007/978-3-662-49665-7_15",

language = "English",

isbn = "978-3-662-49664-0",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "251--268",

editor = "Perdita Stevens and Andzej Wasowski",

booktitle = "Fundamental Approaches to Software Engineering",

}

Gerhold, M & Stoelinga, MIA 2016, Model-Based Testing of Probabilistic Systems. in P Stevens & A Wasowski (eds), Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. Lecture Notes in Computer Science, vol. 9633, Springer, Heidelberg-Berlin, pp. 251-268, 19th International Conference on Fundamental Approaches to Software Engineering, FASE 2016, Eindhoven, Netherlands, 2/04/16. https://doi.org/10.1007/978-3-662-49665-7_15

Model-Based Testing of Probabilistic Systems. / Gerhold, Marcus ; Stoelinga, Mariëlle Ida Antoinette.

Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. ed. / Perdita Stevens; Andzej Wasowski. Heidelberg-Berlin : Springer, 2016. p. 251-268 (Lecture Notes in Computer Science; Vol. 9633).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Model-Based Testing of Probabilistic Systems

AU - Gerhold, Marcus

AU - Stoelinga, Mariëlle Ida Antoinette

PY - 2016/4

Y1 - 2016/4

N2 - This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

AB - This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

KW - FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS

KW - EC Grant Agreement nr.: FP7/318490

KW - Statistical hypothesis testing

KW - Trace distribution

KW - False rejection

KW - Probabilistic automaton

KW - Finite path

U2 - 10.1007/978-3-662-49665-7_15

DO - 10.1007/978-3-662-49665-7_15

M3 - Conference contribution

SN - 978-3-662-49664-0

T3 - Lecture Notes in Computer Science

SP - 251

EP - 268

BT - Fundamental Approaches to Software Engineering

A2 - Stevens, Perdita

A2 - Wasowski, Andzej

PB - Springer

CY - Heidelberg-Berlin

ER -

Gerhold M , Stoelinga MIA. Model-Based Testing of Probabilistic Systems. In Stevens P, Wasowski A, editors, Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. Heidelberg-Berlin: Springer. 2016. p. 251-268. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-662-49665-7_15