Model-based testing of stochastically timed systems

Marcus Gerhold, Arnd Hartmanns* (Corresponding Author), Mariëlle Stoelinga

*Corresponding author for this work

    Research output: Contribution to journal › Article › Academic › peer-review

    13 Downloads (Pure)

    Abstract

    Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Classical model-based testing is insufficient for such systems: it only covers functional correctness. In this paper, we present two model-based testing frameworks that additionally cover the stochastic aspects in hard and soft real-time systems. Using the theory of Markov automata and stochastic automata for specifications, test cases, and a formal notion of conformance, they provide clean mechanisms to represent underspecification, randomisation, and stochastic timing. Markov automata provide a simple memoryless model of time, while stochastic automata support arbitrary continuous and discrete probability distributions. We cleanly define the theoretical foundations, outline practical algorithms for statistical conformance checking, and evaluate both frameworks’ capabilities by testing timing aspects of the Bluetooth device discovery protocol. We highlight the trade-off of simple and efficient statistical evaluation for Markov automata versus precise and realistic modelling with stochastic automata.
    Original languageEnglish
    Pages (from-to)207-233
    Number of pages27
    JournalInnovations in systems and software engineering
    Volume15
    Issue number3-4
    Early online date18 Jun 2019
    DOIs
    Publication statusPublished - Sep 2019

    Fingerprint

    Testing
    Bluetooth
    Real time systems
    Probability distributions
    Specifications
    Network protocols

    Keywords

    • UT-Hybrid-D

    Cite this

    Gerhold, Marcus ; Hartmanns, Arnd ; Stoelinga, Mariëlle. / Model-based testing of stochastically timed systems. In: Innovations in systems and software engineering. 2019 ; Vol. 15, No. 3-4. pp. 207-233.
    @article{4b9442a19eb14100bf2fc1d95bd16813,
    title = "Model-based testing of stochastically timed systems",
    abstract = "Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Classical model-based testing is insufficient for such systems: it only covers functional correctness. In this paper, we present two model-based testing frameworks that additionally cover the stochastic aspects in hard and soft real-time systems. Using the theory of Markov automata and stochastic automata for specifications, test cases, and a formal notion of conformance, they provide clean mechanisms to represent underspecification, randomisation, and stochastic timing. Markov automata provide a simple memoryless model of time, while stochastic automata support arbitrary continuous and discrete probability distributions. We cleanly define the theoretical foundations, outline practical algorithms for statistical conformance checking, and evaluate both frameworks’ capabilities by testing timing aspects of the Bluetooth device discovery protocol. We highlight the trade-off of simple and efficient statistical evaluation for Markov automata versus precise and realistic modelling with stochastic automata.",
    keywords = "UT-Hybrid-D",
    author = "Marcus Gerhold and Arnd Hartmanns and Mari{\"e}lle Stoelinga",
    note = "Springer deal",
    year = "2019",
    month = "9",
    doi = "10.1007/s11334-019-00349-z",
    language = "English",
    volume = "15",
    pages = "207--233",
    journal = "Innovations in systems and software engineering",
    issn = "1614-5046",
    publisher = "Springer",
    number = "3-4",

    }

    Gerhold, M, Hartmanns, A & Stoelinga, M 2019, 'Model-based testing of stochastically timed systems', Innovations in systems and software engineering, vol. 15, no. 3-4, pp. 207-233. https://doi.org/10.1007/s11334-019-00349-z

    Model-based testing of stochastically timed systems. / Gerhold, Marcus; Hartmanns, Arnd (Corresponding Author); Stoelinga, Mariëlle.

    In: Innovations in systems and software engineering, Vol. 15, No. 3-4, 09.2019, p. 207-233.

    Research output: Contribution to journal › Article › Academic › peer-review

    TY - JOUR

    T1 - Model-based testing of stochastically timed systems

    AU - Gerhold, Marcus

    AU - Hartmanns, Arnd

    AU - Stoelinga, Mariëlle

    N1 - Springer deal

    PY - 2019/9

    Y1 - 2019/9

    N2 - Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Classical model-based testing is insufficient for such systems: it only covers functional correctness. In this paper, we present two model-based testing frameworks that additionally cover the stochastic aspects in hard and soft real-time systems. Using the theory of Markov automata and stochastic automata for specifications, test cases, and a formal notion of conformance, they provide clean mechanisms to represent underspecification, randomisation, and stochastic timing. Markov automata provide a simple memoryless model of time, while stochastic automata support arbitrary continuous and discrete probability distributions. We cleanly define the theoretical foundations, outline practical algorithms for statistical conformance checking, and evaluate both frameworks’ capabilities by testing timing aspects of the Bluetooth device discovery protocol. We highlight the trade-off of simple and efficient statistical evaluation for Markov automata versus precise and realistic modelling with stochastic automata.

    AB - Many systems are inherently stochastic: they interact with unpredictable environments or use randomised algorithms. Classical model-based testing is insufficient for such systems: it only covers functional correctness. In this paper, we present two model-based testing frameworks that additionally cover the stochastic aspects in hard and soft real-time systems. Using the theory of Markov automata and stochastic automata for specifications, test cases, and a formal notion of conformance, they provide clean mechanisms to represent underspecification, randomisation, and stochastic timing. Markov automata provide a simple memoryless model of time, while stochastic automata support arbitrary continuous and discrete probability distributions. We cleanly define the theoretical foundations, outline practical algorithms for statistical conformance checking, and evaluate both frameworks’ capabilities by testing timing aspects of the Bluetooth device discovery protocol. We highlight the trade-off of simple and efficient statistical evaluation for Markov automata versus precise and realistic modelling with stochastic automata.

    KW - UT-Hybrid-D

    U2 - 10.1007/s11334-019-00349-z

    DO - 10.1007/s11334-019-00349-z

    M3 - Article

    VL - 15

    SP - 207

    EP - 233

    JO - Innovations in systems and software engineering

    JF - Innovations in systems and software engineering

    SN - 1614-5046

    IS - 3-4

    ER -

    Gerhold M, Hartmanns A, Stoelinga M. Model-based testing of stochastically timed systems. Innovations in systems and software engineering. 2019 Sep;15(3-4):207-233. https://doi.org/10.1007/s11334-019-00349-z

    Access to Document