Model-Based Testing of Probabilistic Systems

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    8 Citations (Scopus)
    89 Downloads (Pure)

    Abstract

    This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.
    Original languageEnglish
    Title of host publicationFundamental Approaches to Software Engineering
    Subtitle of host publication19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings
    EditorsPerdita Stevens, Andzej Wasowski
    Place of PublicationHeidelberg-Berlin
    PublisherSpringer
    Pages251-268
    Number of pages18
    ISBN (Electronic)978-3-662-49665-7
    ISBN (Print)978-3-662-49664-0
    DOIs
    Publication statusPublished - Apr 2016
    Event19th International Conference on Fundamental Approaches to Software Engineering 2016: Proceedings of the 19th International Conference, Fundamental Approaches to Software Engineering, FASE 2016 - Eindhoven University of Technology, Eindhoven, Netherlands
    Duration: 2 Apr 20168 Apr 2016
    Conference number: 19
    https://www.etaps.org/index.php/2016/fase

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer Verlag
    Volume9633
    ISSN (Print)0302-9743
    ISSN (Electronic)1611-3349

    Conference

    Conference19th International Conference on Fundamental Approaches to Software Engineering 2016
    Abbreviated titleFASE 2016
    CountryNetherlands
    CityEindhoven
    Period2/04/168/04/16
    Internet address

    Fingerprint

    Testing
    Statistical methods
    Specifications

    Keywords

    • FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS
    • EWI-27400
    • METIS-320888
    • IR-102862
    • EC Grant Agreement nr.: FP7/318490

    Cite this

    Gerhold, M., & Stoelinga, M. I. A. (2016). Model-Based Testing of Probabilistic Systems. In P. Stevens, & A. Wasowski (Eds.), Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings (pp. 251-268). (Lecture Notes in Computer Science; Vol. 9633). Heidelberg-Berlin: Springer. https://doi.org/10.1007/978-3-662-49665-7_15
    Gerhold, Marcus ; Stoelinga, Mariëlle Ida Antoinette. / Model-Based Testing of Probabilistic Systems. Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. editor / Perdita Stevens ; Andzej Wasowski. Heidelberg-Berlin : Springer, 2016. pp. 251-268 (Lecture Notes in Computer Science).
    @inproceedings{672e674a112047888e8102eddd39f562,
    title = "Model-Based Testing of Probabilistic Systems",
    abstract = "This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.",
    keywords = "FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS, EWI-27400, METIS-320888, IR-102862, EC Grant Agreement nr.: FP7/318490",
    author = "Marcus Gerhold and Stoelinga, {Mari{\"e}lle Ida Antoinette}",
    year = "2016",
    month = "4",
    doi = "10.1007/978-3-662-49665-7_15",
    language = "English",
    isbn = "978-3-662-49664-0",
    series = "Lecture Notes in Computer Science",
    publisher = "Springer",
    pages = "251--268",
    editor = "Perdita Stevens and Andzej Wasowski",
    booktitle = "Fundamental Approaches to Software Engineering",

    }

    Gerhold, M & Stoelinga, MIA 2016, Model-Based Testing of Probabilistic Systems. in P Stevens & A Wasowski (eds), Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. Lecture Notes in Computer Science, vol. 9633, Springer, Heidelberg-Berlin, pp. 251-268, 19th International Conference on Fundamental Approaches to Software Engineering 2016, Eindhoven, Netherlands, 2/04/16. https://doi.org/10.1007/978-3-662-49665-7_15

    Model-Based Testing of Probabilistic Systems. / Gerhold, Marcus; Stoelinga, Mariëlle Ida Antoinette.

    Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. ed. / Perdita Stevens; Andzej Wasowski. Heidelberg-Berlin : Springer, 2016. p. 251-268 (Lecture Notes in Computer Science; Vol. 9633).

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    TY - GEN

    T1 - Model-Based Testing of Probabilistic Systems

    AU - Gerhold, Marcus

    AU - Stoelinga, Mariëlle Ida Antoinette

    PY - 2016/4

    Y1 - 2016/4

    N2 - This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

    AB - This paper presents a model-based testing framework for probabilistic systems. We provide algorithms to generate, execute and evaluate test cases from a probabilistic requirements model. In doing so, we connect ioco-theory for model-based testing and statistical hypothesis testing: our ioco-style algorithms handle the functional aspects, while statistical methods, using χ2χ2 tests and fitting functions, assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Key results of our paper are the classical soundness and completeness properties, establishing the mathematical correctness of our framework; Soundness states that each test case is assigned the right verdict. Completeness states that the framework is powerful enough to discover each probabilistic deviation from the specification, with arbitrary precision. We illustrate the use of our framework via two case studies.

    KW - FMT-FMPA: FORMAL METHODS FOR PERFORMANCE ANALYSIS

    KW - EWI-27400

    KW - METIS-320888

    KW - IR-102862

    KW - EC Grant Agreement nr.: FP7/318490

    U2 - 10.1007/978-3-662-49665-7_15

    DO - 10.1007/978-3-662-49665-7_15

    M3 - Conference contribution

    SN - 978-3-662-49664-0

    T3 - Lecture Notes in Computer Science

    SP - 251

    EP - 268

    BT - Fundamental Approaches to Software Engineering

    A2 - Stevens, Perdita

    A2 - Wasowski, Andzej

    PB - Springer

    CY - Heidelberg-Berlin

    ER -

    Gerhold M, Stoelinga MIA. Model-Based Testing of Probabilistic Systems. In Stevens P, Wasowski A, editors, Fundamental Approaches to Software Engineering: 19th International Conference, FASE 2016, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2016, Eindhoven, The Netherlands, April 2–8, 2016, Proceedings. Heidelberg-Berlin: Springer. 2016. p. 251-268. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-662-49665-7_15