A Framework for the Evaluation of Semantics-based Service Composition Approaches

Eduardo Goncalves da Silva, Luis Ferreira Pires, Marten J. van Sinderen

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    5 Citations (Scopus)
    70 Downloads (Pure)

    Abstract

    The benefits of service composition are being largely acknowledged in the literature nowadays. However, as the amount of available services increases, it becomes difficult to manage, discover, select and compose them, so that automation is required in these processes. This can be achieved by using semantic information represented in ontologies. Currently there are many different approaches that support semantics-based service composition. However, still little effort has been spent on creating a common methodology to evaluate and compare such approaches. In this paper we present our initial ideas to create an evaluation framework for semantics-based service composition approaches. We use a collection of existing services, and define a set of evaluation metrics, confusion matrix-based and time-based. Furthermore, we present how composition evaluation scenarios are generated from the collection of services and specify the strategy to be used in the evaluation process. We demonstrate the proposed framework through an example. Currently there are mechanisms and initiatives to address the evaluation of the semantics-based service discovery and matchmaking approaches. However, still few efforts have been spent on the creation of comprehensive evaluation mechanisms for semantics-based service composition approaches.
    Original languageUndefined
    Title of host publicationProceeding of IEEE European Conference on Web Services, ECOWS 2009
    EditorsR. Eshuis, P.W.P.J. Grefen, P. Grefen, G. A. Papadopoulos
    Place of PublicationLos Alamitos
    PublisherIEEE Computer Society
    Pages66-74
    Number of pages9
    ISBN (Print)978-0-7695-3854-9
    DOIs
    Publication statusPublished - Nov 2009

    Publication series

    Name
    PublisherIEEE Computer Society Press

    Keywords

    • IR-68833
    • METIS-264205
    • Service oriented computing
    • EWI-16939
    • Evaluation Framework
    • SCS-Services
    • Semantic Services

    Cite this

    Goncalves da Silva, E., Ferreira Pires, L., & van Sinderen, M. J. (2009). A Framework for the Evaluation of Semantics-based Service Composition Approaches. In R. Eshuis, P. W. P. J. Grefen, P. Grefen, & G. A. Papadopoulos (Eds.), Proceeding of IEEE European Conference on Web Services, ECOWS 2009 (pp. 66-74). [10.1109/ECOWS.2009.23] Los Alamitos: IEEE Computer Society. https://doi.org/10.1109/ECOWS.2009.23