Model-Based Testing of Probabilistic Systems

  • 7 Citations

Abstract

This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.
Original languageEnglish
Title of host publicationFundamental Approaches to Software Engineering
Subtitle of host publication19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings
EditorsPerdita Stevens, Andzej Wasowski
Place of PublicationHeidelberg-Berlin
PublisherSpringer Verlag
Pages251-268
Number of pages18
ISBN (Electronic)978-3-662-49665-7
ISBN (Print)978-3-662-49664-0
DOIs
StatePublished - Apr 2016
Event19th International Conference on Fundamental Approaches to Software Engineering 2016 - Eindhoven, Netherlands

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Verlag
Volume9633
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference19th International Conference on Fundamental Approaches to Software Engineering 2016
Abbreviated titleFASE 2016
CountryNetherlands
CityEindhoven
Period2/04/168/04/16
Internet address

Fingerprint

Testing
Statistical methods
Specifications

Keywords

  • FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS
  • EWI-27400
  • METIS-320888
  • IR-102862
  • EC Grant Agreement nr.: FP7/318490

Cite this

Gerhold, M., & Stoelinga, M. I. A. (2016). Model-Based Testing of Probabilistic Systems. In P. Stevens, & A. Wasowski (Eds.), Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings (pp. 251-268). (Lecture Notes in Computer Science; Vol. 9633). Heidelberg-Berlin: Springer Verlag. DOI: 10.1007/978-3-662-49665-7_15

Gerhold, Marcus; Stoelinga, Mariëlle Ida Antoinette / Model-Based Testing of Probabilistic Systems.

Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. ed. / Perdita Stevens; Andzej Wasowski. Heidelberg-Berlin : Springer Verlag, 2016. p. 251-268 (Lecture Notes in Computer Science; Vol. 9633).

Research output: Scientific - peer-reviewConference contribution

@inbook{672e674a112047888e8102eddd39f562,
title = "Model-Based Testing of Probabilistic Systems",
abstract = "This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.",
keywords = "FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS, EWI-27400, METIS-320888, IR-102862, EC Grant Agreement nr.: FP7/318490",
author = "Marcus Gerhold and Stoelinga, {Mariëlle Ida Antoinette}",
year = "2016",
month = "4",
doi = "10.1007/978-3-662-49665-7_15",
isbn = "978-3-662-49664-0",
series = "Lecture Notes in Computer Science",
publisher = "Springer Verlag",
pages = "251--268",
editor = "Perdita Stevens and Andzej Wasowski",
booktitle = "Fundamental Approaches to Software Engineering",
address = "Germany",

}

Gerhold, M & Stoelinga, MIA 2016, Model-Based Testing of Probabilistic Systems. in P Stevens & A Wasowski (eds), Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. Lecture Notes in Computer Science, vol. 9633, Springer Verlag, Heidelberg-Berlin, pp. 251-268, 19th International Conference on Fundamental Approaches to Software Engineering 2016, Eindhoven, Netherlands, 2-8 April. DOI: 10.1007/978-3-662-49665-7_15

Model-Based Testing of Probabilistic Systems. / Gerhold, Marcus; Stoelinga, Mariëlle Ida Antoinette.

Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. ed. / Perdita Stevens; Andzej Wasowski. Heidelberg-Berlin : Springer Verlag, 2016. p. 251-268 (Lecture Notes in Computer Science; Vol. 9633).

Research output: Scientific - peer-reviewConference contribution

TY - CHAP

T1 - Model-Based Testing of Probabilistic Systems

AU - Gerhold,Marcus

AU - Stoelinga,Mariëlle Ida Antoinette

PY - 2016/4

Y1 - 2016/4

N2 - This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

AB - This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

KW - FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS

KW - EWI-27400

KW - METIS-320888

KW - IR-102862

KW - EC Grant Agreement nr.: FP7/318490

U2 - 10.1007/978-3-662-49665-7_15

DO - 10.1007/978-3-662-49665-7_15

M3 - Conference contribution

SN - 978-3-662-49664-0

T3 - Lecture Notes in Computer Science

SP - 251

EP - 268

BT - Fundamental Approaches to Software Engineering

PB - Springer Verlag

ER -

Gerhold M, Stoelinga MIA. Model-Based Testing of Probabilistic Systems. In Stevens P, Wasowski A, editors, Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. Heidelberg-Berlin: Springer Verlag. 2016. p. 251-268. (Lecture Notes in Computer Science). Available from, DOI: 10.1007/978-3-662-49665-7_15