Quantitative penetration testing with item response theory

Florian Arnold, Wolter Pieters, Mariëlle Stoelinga

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    8 Citations (Scopus)
    12 Downloads (Pure)


    Existing penetration testing approaches assess the vulnerability of a system by determining whether certain attack paths are possible in practice. Thus, penetration testing has so far been used as a qualitative research method. To enable quantitative approaches to security risk management, including decision support based on the cost-effectiveness of countermeasures, one needs quantitative measures of the feasibility of an attack. Also, when physical or social attack steps are involved, the binary view on whether a vulnerability is present or not is insufficient, and one needs some viability metric. When penetration tests are performed anyway, it is very easy for the testers to keep track of, for example, the time they spend on each attack step. Therefore, this paper proposes the concept of quantitative penetration testing to determine the difficulty rather than the possibility of attacks based on such measurements. We do this by step-wise updates of expected time and probability of success for all steps in an attack scenario. In addition, we show how the skill of the testers can be included to improve the accuracy of the metrics, based on the framework of item response theory (Elo ratings). We prove the feasibility of the approach by means of simulations, and discuss application possibilities.
    Original languageEnglish
    Title of host publication9th International Conference on Information Assurance and Security, IAS 2013
    Place of PublicationPiscataway, NJ
    Number of pages6
    ISBN (Print)978-1-4799-2989-4
    Publication statusPublished - 6 Dec 2013
    Event9th International Conference on Information Assurance and Security, IAS 2013 - Gammarth, Tunisia
    Duration: 4 Dec 20136 Dec 2013
    Conference number: 9


    Conference9th International Conference on Information Assurance and Security, IAS 2013
    Abbreviated titleIAS


    • quantitative security
    • EWI-25270
    • EC Grant Agreement nr.: FP7/2007-2013
    • EC Grant Agreement nr.: FP7/318003
    • IR-92537
    • Socio-technical security
    • Security Metrics
    • Item Response Theory
    • METIS-309645
    • Penetration Testing


    Dive into the research topics of 'Quantitative penetration testing with item response theory'. Together they form a unique fingerprint.

    Cite this