Model-Based Testing of Probabilistic Systems with Stochastic Time

Research output: Chapter in Book/Report/Conference proceedingConference contribution

  • 1 Citations

Abstract

This paper presents a model-based testing framework for black-box probabilistic systems with stochastic continuous time. Markov automata are used as an underlying model. We show how to generate, execute and evaluate test cases automatically from a probabilistically timed requirements model. In doing so, we connect classical ioco-theory with statistical hypothesis testing; our ioco-style algorithms test for functional behaviour, while χ2 hypothesis tests and confidence interval estimations assess the statistical correctness of the system.

A crucial development are the classical soundness and completeness properties of our framework. Soundness states that test cases assign the correct verdict, while completeness states that our methods are powerful enough to discover each discrepancy in functional or statistical misbehaviour, up to arbitrary precision.

We illustrate our framework via the Bluetooth device discovery protocol.
LanguageEnglish
Title of host publicationTests and Proofs
Subtitle of host publication11th International Conference, TAP 2017, Held as Part of STAF 2017, Marburg, Germany, July 19–20, 2017, Proceedings
EditorsEinar Broch Johnsen
Pages77-97
ISBN (Electronic)978-3-319-61467-0
DOIs
StatePublished - 18 Jun 2017
Event11th International Conference on Tests & Proofs - Marburg, Germany
Duration: 19 Jul 201720 Jul 2017
Conference number: 11
http://www.seceng.informatik.tu-darmstadt.de/tap2017/

Publication series

NameLecture notes in computer science
PublisherSpringer
Volume10375

Conference

Conference11th International Conference on Tests & Proofs
Abbreviated titleTAP 2017
CountryGermany
CityMarburg
Period19/07/1720/07/17
Internet address

Fingerprint

Testing
Bluetooth
Network protocols

Cite this

Gerhold, M., & Stoelinga, M. (2017). Model-Based Testing of Probabilistic Systems with Stochastic Time. In E. Broch Johnsen (Ed.), Tests and Proofs: 11th International Conference, TAP 2017, Held as Part of STAF 2017, Marburg, Germany, July 19–20, 2017, Proceedings (pp. 77-97). (Lecture notes in computer science; Vol. 10375). DOI: 10.1007/978-3-319-61467-0_5
Gerhold, Marcus ; Stoelinga, Mariëlle. / Model-Based Testing of Probabilistic Systems with Stochastic Time. Tests and Proofs: 11th International Conference, TAP 2017, Held as Part of STAF 2017, Marburg, Germany, July 19–20, 2017, Proceedings. editor / Einar Broch Johnsen. 2017. pp. 77-97 (Lecture notes in computer science).
@inproceedings{f34224e63f6e4a47920203000ab8f0f4,
title = "Model-Based Testing of Probabilistic Systems with Stochastic Time",
abstract = "This paper presents a model-based testing framework for black-box probabilistic systems with stochastic continuous time. Markov automata are used as an underlying model. We show how to generate, execute and evaluate test cases automatically from a probabilistically timed requirements model. In doing so, we connect classical ioco-theory with statistical hypothesis testing; our ioco-style algorithms test for functional behaviour, while χ2 hypothesis tests and confidence interval estimations assess the statistical correctness of the system.A crucial development are the classical soundness and completeness properties of our framework. Soundness states that test cases assign the correct verdict, while completeness states that our methods are powerful enough to discover each discrepancy in functional or statistical misbehaviour, up to arbitrary precision.We illustrate our framework via the Bluetooth device discovery protocol.",
author = "Marcus Gerhold and Mari{\"e}lle Stoelinga",
year = "2017",
month = "6",
day = "18",
doi = "10.1007/978-3-319-61467-0_5",
language = "English",
isbn = "978-3-319-61466-3",
series = "Lecture notes in computer science",
publisher = "Springer",
pages = "77--97",
editor = "{Broch Johnsen}, Einar",
booktitle = "Tests and Proofs",

}

Gerhold, M & Stoelinga, M 2017, Model-Based Testing of Probabilistic Systems with Stochastic Time. in E Broch Johnsen (ed.), Tests and Proofs: 11th International Conference, TAP 2017, Held as Part of STAF 2017, Marburg, Germany, July 19–20, 2017, Proceedings. Lecture notes in computer science, vol. 10375, pp. 77-97, 11th International Conference on Tests & Proofs, Marburg, Germany, 19/07/17. DOI: 10.1007/978-3-319-61467-0_5

Model-Based Testing of Probabilistic Systems with Stochastic Time. / Gerhold, Marcus; Stoelinga, Mariëlle.

Tests and Proofs: 11th International Conference, TAP 2017, Held as Part of STAF 2017, Marburg, Germany, July 19–20, 2017, Proceedings. ed. / Einar Broch Johnsen. 2017. p. 77-97 (Lecture notes in computer science; Vol. 10375).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Model-Based Testing of Probabilistic Systems with Stochastic Time

AU - Gerhold,Marcus

AU - Stoelinga,Mariëlle

PY - 2017/6/18

Y1 - 2017/6/18

N2 - This paper presents a model-based testing framework for black-box probabilistic systems with stochastic continuous time. Markov automata are used as an underlying model. We show how to generate, execute and evaluate test cases automatically from a probabilistically timed requirements model. In doing so, we connect classical ioco-theory with statistical hypothesis testing; our ioco-style algorithms test for functional behaviour, while χ2 hypothesis tests and confidence interval estimations assess the statistical correctness of the system.A crucial development are the classical soundness and completeness properties of our framework. Soundness states that test cases assign the correct verdict, while completeness states that our methods are powerful enough to discover each discrepancy in functional or statistical misbehaviour, up to arbitrary precision.We illustrate our framework via the Bluetooth device discovery protocol.

AB - This paper presents a model-based testing framework for black-box probabilistic systems with stochastic continuous time. Markov automata are used as an underlying model. We show how to generate, execute and evaluate test cases automatically from a probabilistically timed requirements model. In doing so, we connect classical ioco-theory with statistical hypothesis testing; our ioco-style algorithms test for functional behaviour, while χ2 hypothesis tests and confidence interval estimations assess the statistical correctness of the system.A crucial development are the classical soundness and completeness properties of our framework. Soundness states that test cases assign the correct verdict, while completeness states that our methods are powerful enough to discover each discrepancy in functional or statistical misbehaviour, up to arbitrary precision.We illustrate our framework via the Bluetooth device discovery protocol.

U2 - 10.1007/978-3-319-61467-0_5

DO - 10.1007/978-3-319-61467-0_5

M3 - Conference contribution

SN - 978-3-319-61466-3

T3 - Lecture notes in computer science

SP - 77

EP - 97

BT - Tests and Proofs

ER -

Gerhold M, Stoelinga M. Model-Based Testing of Probabilistic Systems with Stochastic Time. In Broch Johnsen E, editor, Tests and Proofs: 11th International Conference, TAP 2017, Held as Part of STAF 2017, Marburg, Germany, July 19–20, 2017, Proceedings. 2017. p. 77-97. (Lecture notes in computer science). Available from, DOI: 10.1007/978-3-319-61467-0_5