A Framework for the Evaluation of Semantics-based Service Composition Approaches

Eduardo Goncalves da Silva, Luis Ferreira Pires, Marten J. van Sinderen

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    5 Citations (Scopus)
    136 Downloads (Pure)


    The benefits of service composition are being largely acknowledged in the literature nowadays. However, as the amount of available services increases, it becomes difficult to manage, discover, select and compose them, so that automation is required in these processes. This can be achieved by using semantic information represented in ontologies. Currently there are many different approaches that support semantics-based service composition. However, still little effort has been spent on creating a common methodology to evaluate and compare such approaches. In this paper we present our initial ideas to create an evaluation framework for semantics-based service composition approaches. We use a collection of existing services, and define a set of evaluation metrics, confusion matrix-based and time-based. Furthermore, we present how composition evaluation scenarios are generated from the collection of services and specify the strategy to be used in the evaluation process. We demonstrate the proposed framework through an example. Currently there are mechanisms and initiatives to address the evaluation of the semantics-based service discovery and matchmaking approaches. However, still few efforts have been spent on the creation of comprehensive evaluation mechanisms for semantics-based service composition approaches.
    Original languageUndefined
    Title of host publicationProceeding of IEEE European Conference on Web Services, ECOWS 2009
    EditorsR. Eshuis, P.W.P.J. Grefen, P. Grefen, G. A. Papadopoulos
    Place of PublicationLos Alamitos
    PublisherIEEE Computer Society
    Number of pages9
    ISBN (Print)978-0-7695-3854-9
    Publication statusPublished - Nov 2009
    EventProceeding of IEEE European Conference on Web Services, ECOWS 2009, Eindhoven: Proceeding of IEEE European Conference on Web Services, ECOWS 2009 - Los Alamitos
    Duration: 1 Nov 2009 → …

    Publication series

    PublisherIEEE Computer Society Press


    ConferenceProceeding of IEEE European Conference on Web Services, ECOWS 2009, Eindhoven
    CityLos Alamitos
    Period1/11/09 → …


    • IR-68833
    • METIS-264205
    • Service oriented computing
    • EWI-16939
    • Evaluation Framework
    • SCS-Services
    • Semantic Services

    Cite this