A Framework for the Evaluation of Semantics-based Service Composition Approaches

Eduardo Goncalves da Silva, Luis Ferreira Pires, Marten J. van Sinderen

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

5 Citations (Scopus)
53 Downloads (Pure)

Abstract

The benefits of service composition are being largely acknowledged in the literature nowadays. However, as the amount of available services increases, it becomes difficult to manage, discover, select and compose them, so that automation is required in these processes. This can be achieved by using semantic information represented in ontologies. Currently there are many different approaches that support semantics-based service composition. However, still little effort has been spent on creating a common methodology to evaluate and compare such approaches. In this paper we present our initial ideas to create an evaluation framework for semantics-based service composition approaches. We use a collection of existing services, and define a set of evaluation metrics, confusion matrix-based and time-based. Furthermore, we present how composition evaluation scenarios are generated from the collection of services and specify the strategy to be used in the evaluation process. We demonstrate the proposed framework through an example. Currently there are mechanisms and initiatives to address the evaluation of the semantics-based service discovery and matchmaking approaches. However, still few efforts have been spent on the creation of comprehensive evaluation mechanisms for semantics-based service composition approaches.
Original languageUndefined
Title of host publicationProceeding of IEEE European Conference on Web Services, ECOWS 2009
EditorsR. Eshuis, P.W.P.J. Grefen, P. Grefen, G. A. Papadopoulos
Place of PublicationLos Alamitos
PublisherIEEE Computer Society
Pages66-74
Number of pages9
ISBN (Print)978-0-7695-3854-9
DOIs
Publication statusPublished - Nov 2009

Publication series

Name
PublisherIEEE Computer Society Press

Keywords

  • IR-68833
  • METIS-264205
  • Service oriented computing
  • EWI-16939
  • Evaluation Framework
  • SCS-Services
  • Semantic Services

Cite this

Goncalves da Silva, E., Ferreira Pires, L., & van Sinderen, M. J. (2009). A Framework for the Evaluation of Semantics-based Service Composition Approaches. In R. Eshuis, P. W. P. J. Grefen, P. Grefen, & G. A. Papadopoulos (Eds.), Proceeding of IEEE European Conference on Web Services, ECOWS 2009 (pp. 66-74). [10.1109/ECOWS.2009.23] Los Alamitos: IEEE Computer Society. https://doi.org/10.1109/ECOWS.2009.23
Goncalves da Silva, Eduardo ; Ferreira Pires, Luis ; van Sinderen, Marten J. / A Framework for the Evaluation of Semantics-based Service Composition Approaches. Proceeding of IEEE European Conference on Web Services, ECOWS 2009. editor / R. Eshuis ; P.W.P.J. Grefen ; P. Grefen ; G. A. Papadopoulos. Los Alamitos : IEEE Computer Society, 2009. pp. 66-74
@inproceedings{ba3fb1c4d2c24b90bfe97b389e13c9f1,
title = "A Framework for the Evaluation of Semantics-based Service Composition Approaches",
abstract = "The benefits of service composition are being largely acknowledged in the literature nowadays. However, as the amount of available services increases, it becomes difficult to manage, discover, select and compose them, so that automation is required in these processes. This can be achieved by using semantic information represented in ontologies. Currently there are many different approaches that support semantics-based service composition. However, still little effort has been spent on creating a common methodology to evaluate and compare such approaches. In this paper we present our initial ideas to create an evaluation framework for semantics-based service composition approaches. We use a collection of existing services, and define a set of evaluation metrics, confusion matrix-based and time-based. Furthermore, we present how composition evaluation scenarios are generated from the collection of services and specify the strategy to be used in the evaluation process. We demonstrate the proposed framework through an example. Currently there are mechanisms and initiatives to address the evaluation of the semantics-based service discovery and matchmaking approaches. However, still few efforts have been spent on the creation of comprehensive evaluation mechanisms for semantics-based service composition approaches.",
keywords = "IR-68833, METIS-264205, Service oriented computing, EWI-16939, Evaluation Framework, SCS-Services, Semantic Services",
author = "{Goncalves da Silva}, Eduardo and {Ferreira Pires}, Luis and {van Sinderen}, {Marten J.}",
note = "10.1109/ECOWS.2009.23",
year = "2009",
month = "11",
doi = "10.1109/ECOWS.2009.23",
language = "Undefined",
isbn = "978-0-7695-3854-9",
publisher = "IEEE Computer Society",
pages = "66--74",
editor = "R. Eshuis and P.W.P.J. Grefen and P. Grefen and Papadopoulos, {G. A.}",
booktitle = "Proceeding of IEEE European Conference on Web Services, ECOWS 2009",
address = "United States",

}

Goncalves da Silva, E, Ferreira Pires, L & van Sinderen, MJ 2009, A Framework for the Evaluation of Semantics-based Service Composition Approaches. in R Eshuis, PWPJ Grefen, P Grefen & GA Papadopoulos (eds), Proceeding of IEEE European Conference on Web Services, ECOWS 2009., 10.1109/ECOWS.2009.23, IEEE Computer Society, Los Alamitos, pp. 66-74. https://doi.org/10.1109/ECOWS.2009.23

A Framework for the Evaluation of Semantics-based Service Composition Approaches. / Goncalves da Silva, Eduardo; Ferreira Pires, Luis; van Sinderen, Marten J.

Proceeding of IEEE European Conference on Web Services, ECOWS 2009. ed. / R. Eshuis; P.W.P.J. Grefen; P. Grefen; G. A. Papadopoulos. Los Alamitos : IEEE Computer Society, 2009. p. 66-74 10.1109/ECOWS.2009.23.

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

TY - GEN

T1 - A Framework for the Evaluation of Semantics-based Service Composition Approaches

AU - Goncalves da Silva, Eduardo

AU - Ferreira Pires, Luis

AU - van Sinderen, Marten J.

N1 - 10.1109/ECOWS.2009.23

PY - 2009/11

Y1 - 2009/11

N2 - The benefits of service composition are being largely acknowledged in the literature nowadays. However, as the amount of available services increases, it becomes difficult to manage, discover, select and compose them, so that automation is required in these processes. This can be achieved by using semantic information represented in ontologies. Currently there are many different approaches that support semantics-based service composition. However, still little effort has been spent on creating a common methodology to evaluate and compare such approaches. In this paper we present our initial ideas to create an evaluation framework for semantics-based service composition approaches. We use a collection of existing services, and define a set of evaluation metrics, confusion matrix-based and time-based. Furthermore, we present how composition evaluation scenarios are generated from the collection of services and specify the strategy to be used in the evaluation process. We demonstrate the proposed framework through an example. Currently there are mechanisms and initiatives to address the evaluation of the semantics-based service discovery and matchmaking approaches. However, still few efforts have been spent on the creation of comprehensive evaluation mechanisms for semantics-based service composition approaches.

AB - The benefits of service composition are being largely acknowledged in the literature nowadays. However, as the amount of available services increases, it becomes difficult to manage, discover, select and compose them, so that automation is required in these processes. This can be achieved by using semantic information represented in ontologies. Currently there are many different approaches that support semantics-based service composition. However, still little effort has been spent on creating a common methodology to evaluate and compare such approaches. In this paper we present our initial ideas to create an evaluation framework for semantics-based service composition approaches. We use a collection of existing services, and define a set of evaluation metrics, confusion matrix-based and time-based. Furthermore, we present how composition evaluation scenarios are generated from the collection of services and specify the strategy to be used in the evaluation process. We demonstrate the proposed framework through an example. Currently there are mechanisms and initiatives to address the evaluation of the semantics-based service discovery and matchmaking approaches. However, still few efforts have been spent on the creation of comprehensive evaluation mechanisms for semantics-based service composition approaches.

KW - IR-68833

KW - METIS-264205

KW - Service oriented computing

KW - EWI-16939

KW - Evaluation Framework

KW - SCS-Services

KW - Semantic Services

U2 - 10.1109/ECOWS.2009.23

DO - 10.1109/ECOWS.2009.23

M3 - Conference contribution

SN - 978-0-7695-3854-9

SP - 66

EP - 74

BT - Proceeding of IEEE European Conference on Web Services, ECOWS 2009

A2 - Eshuis, R.

A2 - Grefen, P.W.P.J.

A2 - Grefen, P.

A2 - Papadopoulos, G. A.

PB - IEEE Computer Society

CY - Los Alamitos

ER -

Goncalves da Silva E, Ferreira Pires L, van Sinderen MJ. A Framework for the Evaluation of Semantics-based Service Composition Approaches. In Eshuis R, Grefen PWPJ, Grefen P, Papadopoulos GA, editors, Proceeding of IEEE European Conference on Web Services, ECOWS 2009. Los Alamitos: IEEE Computer Society. 2009. p. 66-74. 10.1109/ECOWS.2009.23 https://doi.org/10.1109/ECOWS.2009.23