Automated performance evaluation of service-oriented systems

Fredericus Gerrit Brand van den Berg

Research output: ThesisPhD Thesis - Research UT, graduation UTAcademic

2 Citations (Scopus)
104 Downloads (Pure)

Abstract

An embedded system is a software-intensive system that has a dedicated function within a larger system, e.g., medical equipment. They have increased significantly in complexity over time and are confronted with stringent cost constraints.
Embedded systems interact with their environments in a time-critical way, making their safety is predominantly determined by their performance. Good performance is hard to achieve, because: (i) performance evaluation is hardly ever an integral part of software engineering; (ii) system architectures are increasingly heterogeneous, parallel and distributed; (iii) systems are often designed for many product families and different configurations; and, (iv) measurements that gain insight in system performance tend to be expensive to obtain.
In this thesis, we consider so-called service(-oriented) systems, a special class of embedded systems. A service system provides services to its environment, accessible via so-called service requests, and generates exactly one service response to each request. These service requests are functionally isolated, but can affect each other's performance due to competition for shared resources. Evaluating the performance of service systems before implementation is hard; It calls for models that combine timing aspects and mechanisms to deal with uncertainty.
We propose a new performance evaluation framework for service systems: The system designer models the performance of a system, yielding a high-level performance model. Under the hood, this model is transformed into an underlying performance model that is evaluated for performance to yield performance results.
The literature reports about many so-called toolsets, a collection of connected tools that automates (part of) the performance evaluation process, that facilitate the performance evaluation process for the system designer.
Many toolsets, however, require the user to have knowledge of formal performance evaluation techniques, provide a model that is not domain specific and return approximate results.The goal of this thesis is to overcome the above shortcomings.
Therefore, we have designed and implemented iDSL, a language and toolchain for performance evaluation of service systems, which makes use of the so-called Modest language and toolchain to evaluate models using advanced model checking algorithms and discrete-event simulations. Modest has been selected because of its process algebra roots, its relatively readable language, and its support for multiple formalisms and ways of analysis.
For accurate results, iDSL uses a new evaluation technique, which yields full latency distributions. For this purpose, iDSL uses an algorithm on top of Modest, in which many probabilistic model checking iterations are performed in a systematic way.
The applicability of iDSL on real systems has been assessed by conducting several case studies on interventional X-ray systems throughout the development of iDSL. Interventional X-ray systems are dependable medical systems that support minimally-invasive surgeries.
In conclusion, iDSL delivers a fully automated performance evaluation chain from a high-level model to visualized performance results for service systems. This approach is not only very efficient, it also brings advanced formal performance evaluation techniques at the fingertips of system designers, without bothering them with the technical details of these.
Original languageEnglish
Awarding Institution
  • University of Twente
Supervisors/Advisors
  • Haverkort, Boudewijn R.H.M., Supervisor
  • J.J.M., Hooman, Supervisor
Thesis sponsors
Award date14 Jun 2017
Place of PublicationEnschede
Publisher
Print ISBNs978-90-365-4359-0
DOIs
Publication statusPublished - 14 Jun 2017

Fingerprint

Computer systems
Embedded systems
Model checking
Gain measurement
Biomedical equipment
X rays
Discrete event simulation
Surgery
Algebra
Software engineering
Costs

Cite this

van den Berg, Fredericus Gerrit Brand. / Automated performance evaluation of service-oriented systems. Enschede : University of Twente, 2017. 266 p.
@phdthesis{b25cb78f5b684bafbc4f4268da6b1af0,
title = "Automated performance evaluation of service-oriented systems",
abstract = "An embedded system is a software-intensive system that has a dedicated function within a larger system, e.g., medical equipment. They have increased significantly in complexity over time and are confronted with stringent cost constraints.Embedded systems interact with their environments in a time-critical way, making their safety is predominantly determined by their performance. Good performance is hard to achieve, because: (i) performance evaluation is hardly ever an integral part of software engineering; (ii) system architectures are increasingly heterogeneous, parallel and distributed; (iii) systems are often designed for many product families and different configurations; and, (iv) measurements that gain insight in system performance tend to be expensive to obtain.In this thesis, we consider so-called service(-oriented) systems, a special class of embedded systems. A service system provides services to its environment, accessible via so-called service requests, and generates exactly one service response to each request. These service requests are functionally isolated, but can affect each other's performance due to competition for shared resources. Evaluating the performance of service systems before implementation is hard; It calls for models that combine timing aspects and mechanisms to deal with uncertainty.We propose a new performance evaluation framework for service systems: The system designer models the performance of a system, yielding a high-level performance model. Under the hood, this model is transformed into an underlying performance model that is evaluated for performance to yield performance results.The literature reports about many so-called toolsets, a collection of connected tools that automates (part of) the performance evaluation process, that facilitate the performance evaluation process for the system designer.Many toolsets, however, require the user to have knowledge of formal performance evaluation techniques, provide a model that is not domain specific and return approximate results.The goal of this thesis is to overcome the above shortcomings.Therefore, we have designed and implemented iDSL, a language and toolchain for performance evaluation of service systems, which makes use of the so-called Modest language and toolchain to evaluate models using advanced model checking algorithms and discrete-event simulations. Modest has been selected because of its process algebra roots, its relatively readable language, and its support for multiple formalisms and ways of analysis.For accurate results, iDSL uses a new evaluation technique, which yields full latency distributions. For this purpose, iDSL uses an algorithm on top of Modest, in which many probabilistic model checking iterations are performed in a systematic way.The applicability of iDSL on real systems has been assessed by conducting several case studies on interventional X-ray systems throughout the development of iDSL. Interventional X-ray systems are dependable medical systems that support minimally-invasive surgeries. In conclusion, iDSL delivers a fully automated performance evaluation chain from a high-level model to visualized performance results for service systems. This approach is not only very efficient, it also brings advanced formal performance evaluation techniques at the fingertips of system designers, without bothering them with the technical details of these.",
author = "{van den Berg}, {Fredericus Gerrit Brand}",
note = "CTIT Ph.D. thesis series no. 17-434",
year = "2017",
month = "6",
day = "14",
doi = "10.3990/1.9789036543590",
language = "English",
isbn = "978-90-365-4359-0",
publisher = "University of Twente",
address = "Netherlands",
school = "University of Twente",

}

Automated performance evaluation of service-oriented systems. / van den Berg, Fredericus Gerrit Brand.

Enschede : University of Twente, 2017. 266 p.

Research output: ThesisPhD Thesis - Research UT, graduation UTAcademic

TY - THES

T1 - Automated performance evaluation of service-oriented systems

AU - van den Berg, Fredericus Gerrit Brand

N1 - CTIT Ph.D. thesis series no. 17-434

PY - 2017/6/14

Y1 - 2017/6/14

N2 - An embedded system is a software-intensive system that has a dedicated function within a larger system, e.g., medical equipment. They have increased significantly in complexity over time and are confronted with stringent cost constraints.Embedded systems interact with their environments in a time-critical way, making their safety is predominantly determined by their performance. Good performance is hard to achieve, because: (i) performance evaluation is hardly ever an integral part of software engineering; (ii) system architectures are increasingly heterogeneous, parallel and distributed; (iii) systems are often designed for many product families and different configurations; and, (iv) measurements that gain insight in system performance tend to be expensive to obtain.In this thesis, we consider so-called service(-oriented) systems, a special class of embedded systems. A service system provides services to its environment, accessible via so-called service requests, and generates exactly one service response to each request. These service requests are functionally isolated, but can affect each other's performance due to competition for shared resources. Evaluating the performance of service systems before implementation is hard; It calls for models that combine timing aspects and mechanisms to deal with uncertainty.We propose a new performance evaluation framework for service systems: The system designer models the performance of a system, yielding a high-level performance model. Under the hood, this model is transformed into an underlying performance model that is evaluated for performance to yield performance results.The literature reports about many so-called toolsets, a collection of connected tools that automates (part of) the performance evaluation process, that facilitate the performance evaluation process for the system designer.Many toolsets, however, require the user to have knowledge of formal performance evaluation techniques, provide a model that is not domain specific and return approximate results.The goal of this thesis is to overcome the above shortcomings.Therefore, we have designed and implemented iDSL, a language and toolchain for performance evaluation of service systems, which makes use of the so-called Modest language and toolchain to evaluate models using advanced model checking algorithms and discrete-event simulations. Modest has been selected because of its process algebra roots, its relatively readable language, and its support for multiple formalisms and ways of analysis.For accurate results, iDSL uses a new evaluation technique, which yields full latency distributions. For this purpose, iDSL uses an algorithm on top of Modest, in which many probabilistic model checking iterations are performed in a systematic way.The applicability of iDSL on real systems has been assessed by conducting several case studies on interventional X-ray systems throughout the development of iDSL. Interventional X-ray systems are dependable medical systems that support minimally-invasive surgeries. In conclusion, iDSL delivers a fully automated performance evaluation chain from a high-level model to visualized performance results for service systems. This approach is not only very efficient, it also brings advanced formal performance evaluation techniques at the fingertips of system designers, without bothering them with the technical details of these.

AB - An embedded system is a software-intensive system that has a dedicated function within a larger system, e.g., medical equipment. They have increased significantly in complexity over time and are confronted with stringent cost constraints.Embedded systems interact with their environments in a time-critical way, making their safety is predominantly determined by their performance. Good performance is hard to achieve, because: (i) performance evaluation is hardly ever an integral part of software engineering; (ii) system architectures are increasingly heterogeneous, parallel and distributed; (iii) systems are often designed for many product families and different configurations; and, (iv) measurements that gain insight in system performance tend to be expensive to obtain.In this thesis, we consider so-called service(-oriented) systems, a special class of embedded systems. A service system provides services to its environment, accessible via so-called service requests, and generates exactly one service response to each request. These service requests are functionally isolated, but can affect each other's performance due to competition for shared resources. Evaluating the performance of service systems before implementation is hard; It calls for models that combine timing aspects and mechanisms to deal with uncertainty.We propose a new performance evaluation framework for service systems: The system designer models the performance of a system, yielding a high-level performance model. Under the hood, this model is transformed into an underlying performance model that is evaluated for performance to yield performance results.The literature reports about many so-called toolsets, a collection of connected tools that automates (part of) the performance evaluation process, that facilitate the performance evaluation process for the system designer.Many toolsets, however, require the user to have knowledge of formal performance evaluation techniques, provide a model that is not domain specific and return approximate results.The goal of this thesis is to overcome the above shortcomings.Therefore, we have designed and implemented iDSL, a language and toolchain for performance evaluation of service systems, which makes use of the so-called Modest language and toolchain to evaluate models using advanced model checking algorithms and discrete-event simulations. Modest has been selected because of its process algebra roots, its relatively readable language, and its support for multiple formalisms and ways of analysis.For accurate results, iDSL uses a new evaluation technique, which yields full latency distributions. For this purpose, iDSL uses an algorithm on top of Modest, in which many probabilistic model checking iterations are performed in a systematic way.The applicability of iDSL on real systems has been assessed by conducting several case studies on interventional X-ray systems throughout the development of iDSL. Interventional X-ray systems are dependable medical systems that support minimally-invasive surgeries. In conclusion, iDSL delivers a fully automated performance evaluation chain from a high-level model to visualized performance results for service systems. This approach is not only very efficient, it also brings advanced formal performance evaluation techniques at the fingertips of system designers, without bothering them with the technical details of these.

U2 - 10.3990/1.9789036543590

DO - 10.3990/1.9789036543590

M3 - PhD Thesis - Research UT, graduation UT

SN - 978-90-365-4359-0

PB - University of Twente

CY - Enschede

ER -