On the organisation of program verification competitions

Marieke Huisman, Vladimir Klebanov, Rosemary Monahan

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    5 Citations (Scopus)
    19 Downloads (Pure)

    Abstract

    In this paper, we discuss the challenges that have to be addressed when organising program verification competitions. Our focus is on competitions for verification systems where the participants both formalise an informally stated requirement and (typically) provide some guidance for the tool to show it. The paper draws its insights from our experiences with organising a program verification competition at FoVeOOS 2011. We discuss in particular the following aspects: challenge selection, on-site versus online organisation, team composition and judging. We conclude with a list of recommendations for future competition organisers.
    Original languageUndefined
    Title of host publication1st International Workshop on Comparative Empirical Evaluation of Reasoning Systems (COMPARE 2012)
    EditorsVladimir Klebanov, Bernhard Beckert, Armin Biere, Geoff Sutcliffe
    Place of PublicationAachen
    PublisherSun SITE Central Europe
    Pages50-59
    Number of pages10
    Publication statusPublished - Jun 2012

    Publication series

    NameCEUR-WS Proceedings
    PublisherSun SITE Central Europe
    Volume873
    ISSN (Print)1613-0073

    Keywords

    • EWI-22675
    • IR-83451
    • METIS-293239

    Cite this

    Huisman, M., Klebanov, V., & Monahan, R. (2012). On the organisation of program verification competitions. In V. Klebanov, B. Beckert, A. Biere, & G. Sutcliffe (Eds.), 1st International Workshop on Comparative Empirical Evaluation of Reasoning Systems (COMPARE 2012) (pp. 50-59). (CEUR-WS Proceedings; Vol. 873). Aachen: Sun SITE Central Europe.