Abstract

This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.
Original languageUndefined
Title of host publicationProceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016
EditorsPerdita Stevens, Andzej Wasowski
Place of PublicationHeidelberg-Berlin
PublisherSpringer Verlag
Pages251-268
Number of pages18
ISBN (Print)978-3-662-49664-0
DOIs
StatePublished - Apr 2016

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Verlag
Volume9633
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Fingerprint

Testing
Statistical methods
Specifications

Keywords

  • FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS
  • EWI-27400
  • METIS-320888
  • IR-102862
  • EC Grant Agreement nr.: FP7/318490

Cite this

Gerhold, M., & Stoelinga, M. I. A. (2016). Model-Based Testing of Probabilistic Systems. In P. Stevens, & A. Wasowski (Eds.), Proceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016 (pp. 251-268). (Lecture Notes in Computer Science; Vol. 9633). Heidelberg-Berlin: Springer Verlag. DOI: 10.1007/978-3-662-49665-7_15

Gerhold, Marcus; Stoelinga, Mariëlle Ida Antoinette / Model-Based Testing of Probabilistic Systems.

Proceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016. ed. / Perdita Stevens; Andzej Wasowski. Heidelberg-Berlin : Springer Verlag, 2016. p. 251-268 (Lecture Notes in Computer Science; Vol. 9633).

Research output: Scientific - peer-reviewConference contribution

@inbook{672e674a112047888e8102eddd39f562,
title = "Model-Based Testing of Probabilistic Systems",
abstract = "This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.",
keywords = "FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS, EWI-27400, METIS-320888, IR-102862, EC Grant Agreement nr.: FP7/318490",
author = "Marcus Gerhold and Stoelinga, {Mariëlle Ida Antoinette}",
note = "10.1007/978-3-662-49665-7_15",
year = "2016",
month = "4",
doi = "10.1007/978-3-662-49665-7_15",
isbn = "978-3-662-49664-0",
series = "Lecture Notes in Computer Science",
publisher = "Springer Verlag",
pages = "251--268",
editor = "Perdita Stevens and Andzej Wasowski",
booktitle = "Proceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016",

}

Gerhold, M & Stoelinga, MIA 2016, Model-Based Testing of Probabilistic Systems. in P Stevens & A Wasowski (eds), Proceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016. Lecture Notes in Computer Science, vol. 9633, Springer Verlag, Heidelberg-Berlin, pp. 251-268. DOI: 10.1007/978-3-662-49665-7_15

Model-Based Testing of Probabilistic Systems. / Gerhold, Marcus; Stoelinga, Mariëlle Ida Antoinette.

Proceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016. ed. / Perdita Stevens; Andzej Wasowski. Heidelberg-Berlin : Springer Verlag, 2016. p. 251-268 (Lecture Notes in Computer Science; Vol. 9633).

Research output: Scientific - peer-reviewConference contribution

TY - CHAP

T1 - Model-Based Testing of Probabilistic Systems

AU - Gerhold,Marcus

AU - Stoelinga,Mariëlle Ida Antoinette

N1 - 10.1007/978-3-662-49665-7_15

PY - 2016/4

Y1 - 2016/4

N2 - This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

AB - This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

KW - FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS

KW - EWI-27400

KW - METIS-320888

KW - IR-102862

KW - EC Grant Agreement nr.: FP7/318490

U2 - 10.1007/978-3-662-49665-7_15

DO - 10.1007/978-3-662-49665-7_15

M3 - Conference contribution

SN - 978-3-662-49664-0

T3 - Lecture Notes in Computer Science

SP - 251

EP - 268

BT - Proceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016

PB - Springer Verlag

ER -

Gerhold M, Stoelinga MIA. Model-Based Testing of Probabilistic Systems. In Stevens P, Wasowski A, editors, Proceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016. Heidelberg-Berlin: Springer Verlag. 2016. p. 251-268. (Lecture Notes in Computer Science). Available from, DOI: 10.1007/978-3-662-49665-7_15