Model-Based Testing for General Stochastic Time

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.
LanguageEnglish
Title of host publicationNASA Formal Methods
Subtitle of host publication10th International Symposium, NFM 2018, Newport News, VA, USA, April 17-19, 2018, Proceedings
EditorsAaron Dutle, Cesar Munoz, Anthony Narkawicz
Pages203-219
ISBN (Electronic)978-3-319-77935-5
DOIs
StatePublished - 11 Mar 2018
Event10th International Symposium on NASA Formal Methods 2018 - Newport News Marriott at City Center, Newport News, United States
Duration: 17 Apr 201819 Apr 2018
Conference number: 10
https://shemesh.larc.nasa.gov/NFM2018/

Publication series

NameLecture notes in computer science
PublisherSpringer
Volume10811

Conference

Conference10th International Symposium on NASA Formal Methods 2018
Abbreviated titleNFM 2018
CountryUnited States
CityNewport News
Period17/04/1819/04/18
Internet address

Fingerprint

Testing
Bluetooth
Real time systems
Probability distributions
Specifications
Network protocols

Cite this

Gerhold, M., Hartmanns, A., & Stoelinga, M. (2018). Model-Based Testing for General Stochastic Time. In A. Dutle, C. Munoz, & A. Narkawicz (Eds.), NASA Formal Methods: 10th International Symposium, NFM 2018, Newport News, VA, USA, April 17-19, 2018, Proceedings (pp. 203-219). (Lecture notes in computer science; Vol. 10811). DOI: 10.1007/978-3-319-77935-5_15
Gerhold, Marcus ; Hartmanns, Arnd ; Stoelinga, Mariëlle. / Model-Based Testing for General Stochastic Time. NASA Formal Methods: 10th International Symposium, NFM 2018, Newport News, VA, USA, April 17-19, 2018, Proceedings. editor / Aaron Dutle ; Cesar Munoz ; Anthony Narkawicz. 2018. pp. 203-219 (Lecture notes in computer science).
@inproceedings{29fa417e09f54294b249c69277212c82,
title = "Model-Based Testing for General Stochastic Time",
abstract = "Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.",
author = "Marcus Gerhold and Arnd Hartmanns and Mari{\"e}lle Stoelinga",
year = "2018",
month = "3",
day = "11",
doi = "10.1007/978-3-319-77935-5_15",
language = "English",
isbn = "978-3-319-77934-8",
series = "Lecture notes in computer science",
publisher = "Springer",
pages = "203--219",
editor = "Aaron Dutle and Cesar Munoz and Anthony Narkawicz",
booktitle = "NASA Formal Methods",

}

Gerhold, M, Hartmanns, A & Stoelinga, M 2018, Model-Based Testing for General Stochastic Time. in A Dutle, C Munoz & A Narkawicz (eds), NASA Formal Methods: 10th International Symposium, NFM 2018, Newport News, VA, USA, April 17-19, 2018, Proceedings. Lecture notes in computer science, vol. 10811, pp. 203-219, 10th International Symposium on NASA Formal Methods 2018, Newport News, United States, 17/04/18. DOI: 10.1007/978-3-319-77935-5_15

Model-Based Testing for General Stochastic Time. / Gerhold, Marcus; Hartmanns, Arnd; Stoelinga, Mariëlle.

NASA Formal Methods: 10th International Symposium, NFM 2018, Newport News, VA, USA, April 17-19, 2018, Proceedings. ed. / Aaron Dutle; Cesar Munoz; Anthony Narkawicz. 2018. p. 203-219 (Lecture notes in computer science; Vol. 10811).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Model-Based Testing for General Stochastic Time

AU - Gerhold,Marcus

AU - Hartmanns,Arnd

AU - Stoelinga,Mariëlle

PY - 2018/3/11

Y1 - 2018/3/11

N2 - Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.

AB - Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.

U2 - 10.1007/978-3-319-77935-5_15

DO - 10.1007/978-3-319-77935-5_15

M3 - Conference contribution

SN - 978-3-319-77934-8

T3 - Lecture notes in computer science

SP - 203

EP - 219

BT - NASA Formal Methods

ER -

Gerhold M, Hartmanns A, Stoelinga M. Model-Based Testing for General Stochastic Time. In Dutle A, Munoz C, Narkawicz A, editors, NASA Formal Methods: 10th International Symposium, NFM 2018, Newport News, VA, USA, April 17-19, 2018, Proceedings. 2018. p. 203-219. (Lecture notes in computer science). Available from, DOI: 10.1007/978-3-319-77935-5_15