Model-Based Testing for General Stochastic Time

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    1 Citation (Scopus)
    15 Downloads (Pure)

    Abstract

    Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.
    Original languageEnglish
    Title of host publicationProceedings of the 10th International NASA Formal Methods Symposium (NFM 2018)
    EditorsAaron Dutle, Cesar Munoz, Anthony Narkawicz
    PublisherSpringer
    Pages203-219
    ISBN (Electronic)978-3-319-77935-5
    ISBN (Print)978-3-319-77934-8
    DOIs
    Publication statusPublished - 11 Mar 2018
    Event10th International Symposium on NASA Formal Methods 2018 - Newport News Marriott at City Center, Newport News, United States
    Duration: 17 Apr 201819 Apr 2018
    Conference number: 10
    https://shemesh.larc.nasa.gov/NFM2018/

    Publication series

    NameLecture Notes in Computer Science
    PublisherSpringer
    Volume10811

    Conference

    Conference10th International Symposium on NASA Formal Methods 2018
    Abbreviated titleNFM 2018
    CountryUnited States
    CityNewport News
    Period17/04/1819/04/18
    Internet address

    Fingerprint

    Testing
    Bluetooth
    Real time systems
    Probability distributions
    Specifications
    Network protocols

    Cite this

    Gerhold, M., Hartmanns, A., & Stoelinga, M. (2018). Model-Based Testing for General Stochastic Time. In A. Dutle, C. Munoz, & A. Narkawicz (Eds.), Proceedings of the 10th International NASA Formal Methods Symposium (NFM 2018) (pp. 203-219). (Lecture Notes in Computer Science; Vol. 10811). Springer. https://doi.org/10.1007/978-3-319-77935-5_15
    Gerhold, Marcus ; Hartmanns, Arnd ; Stoelinga, Mariëlle. / Model-Based Testing for General Stochastic Time. Proceedings of the 10th International NASA Formal Methods Symposium (NFM 2018). editor / Aaron Dutle ; Cesar Munoz ; Anthony Narkawicz. Springer, 2018. pp. 203-219 (Lecture Notes in Computer Science).
    @inproceedings{29fa417e09f54294b249c69277212c82,
    title = "Model-Based Testing for General Stochastic Time",
    abstract = "Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.",
    author = "Marcus Gerhold and Arnd Hartmanns and Mari{\"e}lle Stoelinga",
    year = "2018",
    month = "3",
    day = "11",
    doi = "10.1007/978-3-319-77935-5_15",
    language = "English",
    isbn = "978-3-319-77934-8",
    series = "Lecture Notes in Computer Science",
    publisher = "Springer",
    pages = "203--219",
    editor = "Aaron Dutle and Cesar Munoz and Anthony Narkawicz",
    booktitle = "Proceedings of the 10th International NASA Formal Methods Symposium (NFM 2018)",

    }

    Gerhold, M, Hartmanns, A & Stoelinga, M 2018, Model-Based Testing for General Stochastic Time. in A Dutle, C Munoz & A Narkawicz (eds), Proceedings of the 10th International NASA Formal Methods Symposium (NFM 2018). Lecture Notes in Computer Science, vol. 10811, Springer, pp. 203-219, 10th International Symposium on NASA Formal Methods 2018, Newport News, United States, 17/04/18. https://doi.org/10.1007/978-3-319-77935-5_15

    Model-Based Testing for General Stochastic Time. / Gerhold, Marcus; Hartmanns, Arnd; Stoelinga, Mariëlle.

    Proceedings of the 10th International NASA Formal Methods Symposium (NFM 2018). ed. / Aaron Dutle; Cesar Munoz; Anthony Narkawicz. Springer, 2018. p. 203-219 (Lecture Notes in Computer Science; Vol. 10811).

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    TY - GEN

    T1 - Model-Based Testing for General Stochastic Time

    AU - Gerhold, Marcus

    AU - Hartmanns, Arnd

    AU - Stoelinga, Mariëlle

    PY - 2018/3/11

    Y1 - 2018/3/11

    N2 - Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.

    AB - Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Then classical model-based testing is insufficient: it only covers functional correctness. In this paper, we present a new model-based testing framework that additionally covers the stochastic aspects in hard and soft real-time systems. Using the theory of stochastic automata for specifications, test cases and a formal notion of conformance, it provides clean mechanisms to represent underspecification, randomisation, and stochastic timing. Supporting arbitrary continuous and discrete probability distributions, the framework generalises previous work based on purely Markovian models. We cleanly define its theoretical foundations, and then outline a practical algorithm for statistical conformance testing based on the Kolmogorov-Smirnov test. We exemplify the framework’s capabilities and tradeoffs by testing timing aspects of the Bluetooth device discovery protocol.

    U2 - 10.1007/978-3-319-77935-5_15

    DO - 10.1007/978-3-319-77935-5_15

    M3 - Conference contribution

    SN - 978-3-319-77934-8

    T3 - Lecture Notes in Computer Science

    SP - 203

    EP - 219

    BT - Proceedings of the 10th International NASA Formal Methods Symposium (NFM 2018)

    A2 - Dutle, Aaron

    A2 - Munoz, Cesar

    A2 - Narkawicz, Anthony

    PB - Springer

    ER -

    Gerhold M, Hartmanns A, Stoelinga M. Model-Based Testing for General Stochastic Time. In Dutle A, Munoz C, Narkawicz A, editors, Proceedings of the 10th International NASA Formal Methods Symposium (NFM 2018). Springer. 2018. p. 203-219. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-319-77935-5_15